Jul 28 19:27:33.931298 ip-10-0-138-37 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 19:27:33.931313 ip-10-0-138-37 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 19:27:33.931324 ip-10-0-138-37 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 19:27:33.931726 ip-10-0-138-37 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 19:27:43.946307 ip-10-0-138-37 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 19:27:43.946323 ip-10-0-138-37 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d7a853c8f6a1442f933505268e476ad6 --
Jul 28 19:29:34.320577 ip-10-0-138-37 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 19:29:34.621706 ip-10-0-138-37 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:29:34.621706 ip-10-0-138-37 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 19:29:34.622282 ip-10-0-138-37 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:29:34.622282 ip-10-0-138-37 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 19:29:34.622282 ip-10-0-138-37 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:29:34.623228 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.623155    2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 19:29:34.626540 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626524    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:29:34.626540 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626540    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626545    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626548    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626551    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626554    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626557    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626559    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626562    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626565    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626568    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626571    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626575    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626579    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626582    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626585    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626588    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626592    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626595    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626598    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:29:34.626608 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626600    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626603    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626605    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626608    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626613    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626616    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626619    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626622    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626624    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626627    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626630    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626632    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626635    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626637    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626640    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626642    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626645    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626648    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626650    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:29:34.627059 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626653    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626655    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626658    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626660    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626663    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626665    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626668    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626671    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626673    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626676    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626679    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626681    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626684    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626687    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626689    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626693    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626696    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626700    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626702    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626705    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:29:34.627533 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626707    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626710    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626712    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626715    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626718    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626720    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626723    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626725    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626727    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626730    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626732    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626735    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626737    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626740    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626743    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626745    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626748    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626751    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626753    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:29:34.628023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626756    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626758    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626761    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626763    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626767    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626769    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626772    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626774    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626778    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626780    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626783    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626785    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626788    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626790    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626793    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626795    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626798    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626801    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626805    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626807    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:29:34.628508 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.626810    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628249    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628257    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628261    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628265    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628268    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628270    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628273    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628276    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628278    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628281    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628283    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628286    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628288    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628291    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628294    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628297    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628300    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628302    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628305    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:29:34.628996 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628308    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628311    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628314    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628316    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628319    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628322    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628325    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628327    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628330    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628332    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628335    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628337    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628340    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628342    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628345    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628347    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628350    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628352    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628355    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:29:34.629504 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628357    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628360    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628362    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628365    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628367    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628370    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628373    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628376    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628379    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628382    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628385    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628388    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628391    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628394    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628396    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628400    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628402    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628405    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628407    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628410    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:29:34.629966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628412    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628415    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628419    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628422    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628425    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628428    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628431    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628434    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628437    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628439    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628442    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628446    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628449    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628453    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628456    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628473    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628478    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628483    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628487    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:29:34.630484 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628492    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628496    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628499    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628501    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628504    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628507    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628511    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628513    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628516    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628519    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628521    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628525    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628528    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628532    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628534    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628538    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628540    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628543    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628546    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:29:34.630966 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628548    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:29:34.631444 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628551    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:29:34.631444 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628554    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:29:34.631444 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628914    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:29:34.631444 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628919    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:29:34.631444 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628923    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:29:34.631444 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628926    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:29:34.631444 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628929    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:29:34.631444 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628933    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:29:34.631444 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628937    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:29:34.631444 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628940    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:29:34.631444 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628943    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:29:34.631444 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628946    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:29:34.631444 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628949    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:29:34.631444 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628952    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:29:34.631444 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628955    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:29:34.631444 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628959    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:29:34.631444 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628962    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:29:34.631444 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628966    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628969    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628972    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628975    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628977    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628980    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628982    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628985    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628988    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628991    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628993    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628996    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.628998    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629001    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629004    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629007    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629010    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629013    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629015    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629019    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:29:34.631887 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629022    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629024    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629026    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629029    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629032    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629034    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629037    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629039    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629041    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629044    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629046    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629049    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629051    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629055    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629057    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629060    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629062    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629077    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629080    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:29:34.632409 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629083    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629086    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629088    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629091    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629094    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629097    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629100    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629102    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629105    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629108    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629112    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629115    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629117    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629120    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629122    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629125    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629128    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629130    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629132    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629136    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:29:34.632855 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629138    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629141    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629143    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629146    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629148    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629151    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629153    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629156    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629159    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629161    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629164    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629167    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629169    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629172    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629174    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629177    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629179    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629182    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629184    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629187    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:29:34.633357 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629189    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629192    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629195    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629197    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629200    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629271    2572 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629286    2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629295    2572 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629301    2572 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629305    2572 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629308    2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629312    2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629316    2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629319    2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629322    2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629325    2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629329    2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629331    2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629335    2572 flags.go:64] FLAG: --cgroup-root=""
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629337    2572 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629341    2572 flags.go:64] FLAG: --client-ca-file=""
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629344    2572 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629347    2572 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 19:29:34.633867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629351    2572 flags.go:64] FLAG: --cluster-domain=""
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629354    2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629358    2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629361    2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629364    2572 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629367    2572 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629372    2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629375    2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629378    2572 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629381    2572 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629386    2572 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629389    2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629392    2572 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629395    2572 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629399    2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629402    2572 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629405    2572 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629408    2572 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629411    2572 flags.go:64] FLAG: --enable-server="true"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629414    2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629418    2572 flags.go:64] FLAG: --event-burst="100"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629421    2572 flags.go:64] FLAG: --event-qps="50"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629424    2572 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629427    2572 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 19:29:34.634464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629430    2572 flags.go:64] FLAG: --eviction-hard=""
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629433    2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629436    2572 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629439    2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629442    2572 flags.go:64] FLAG: --eviction-soft=""
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629445    2572 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629449    2572 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629452    2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629454    2572 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629457    2572 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629460    2572 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629463    2572 flags.go:64] FLAG: --feature-gates=""
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629466    2572 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629469    2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629474    2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629477    2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629480    2572 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629483    2572 flags.go:64] FLAG: --help="false"
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629487    2572 flags.go:64] FLAG: --hostname-override="ip-10-0-138-37.ec2.internal"
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629490    2572 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629493    2572 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629496    2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629500    2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 19:29:34.635057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629503    2572 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629507    2572 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629510    2572 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629512    2572 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629515    2572 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629518    2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629522    2572 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629524    2572 flags.go:64] FLAG: --kube-reserved=""
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629527    2572 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629530    2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629533    2572 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629536    2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629539    2572 flags.go:64] FLAG: --lock-file=""
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629542    2572 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629545    2572 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629548    2572 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629554    2572 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629557    2572 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629560    2572 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629563    2572 flags.go:64] FLAG: --logging-format="text"
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629565    2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629569    2572 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629571    2572 flags.go:64] FLAG: --manifest-url=""
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629576    2572 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629580    2572 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 19:29:34.635736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629583    2572 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629587    2572 flags.go:64] FLAG: --max-pods="110"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629591    2572 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629594    2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629597    2572 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629600    2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629603    2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629606    2572 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629609    2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629616    2572 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629619    2572 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629622    2572 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629625    2572 flags.go:64] FLAG: --pod-cidr=""
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629628    2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629633    2572 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629636    2572 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629639    2572 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629642    2572 flags.go:64] FLAG: --port="10250"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629645    2572 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629648    2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0497049beb4abc19d"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629651    2572 flags.go:64] FLAG: --qos-reserved=""
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629654    2572 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629657    2572 flags.go:64] FLAG: --register-node="true"
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629660    2572 flags.go:64] FLAG: --register-with-taints=""
Jul 28 19:29:34.636512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629663    2572 flags.go:64] FLAG: --registry-burst="10"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629666    2572 flags.go:64] FLAG: --registry-qps="5"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629669    2572 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629672    2572 flags.go:64] FLAG: --reserved-memory=""
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629678    2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629681    2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629684    2572 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629688    2572 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629691    2572 flags.go:64] FLAG: --runonce="false"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629694    2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629698    2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629701    2572 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629704    2572 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629707    2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629710    2572 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629713    2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629716    2572 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629718    2572 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629721    2572 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629724    2572 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629728    2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629730    2572 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629733    2572 flags.go:64] FLAG: --system-cgroups=""
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629736    2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629741    2572 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 19:29:34.637160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629744    2572 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629747    2572 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629751    2572 flags.go:64] FLAG: --tls-min-version=""
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629754    2572 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629757    2572 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629759    2572 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629762    2572 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629765    2572 flags.go:64] FLAG: --v="2"
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629769    2572 flags.go:64] FLAG: --version="false"
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629773    2572 flags.go:64] FLAG: --vmodule=""
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629777    2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.629780    2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629893    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629897    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629902    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629905    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629907    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629911    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629914    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629916    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629919    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629922    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:29:34.638002 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629924    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629927    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629929    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629932    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629934    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629937    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629939    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629942    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629945    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629947    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629950    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629952    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629955    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629957    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629960    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629963    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629965    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629968    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629971    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629974    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:29:34.638593 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629977    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629979    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629982    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629984    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629988    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629991    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629993    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629997    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.629999    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630002    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630004    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630007    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630010    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630013    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630016    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630018    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630021    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630023    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630026    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:29:34.639169 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630028    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630032    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630036    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630039    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630042    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630045    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630050    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630052    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630055    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630058    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630060    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630063    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630078    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630082    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630084    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630087    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630090    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630094    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630097    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630099    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:29:34.639628 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630103    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630105    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630108    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630111    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630113    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630116    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630118    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630121    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630124    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630126    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630129    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630131    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630134    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630137    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630139    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630142    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630144    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630147    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630150    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630152    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:29:34.640134 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630155    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:29:34.640614 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630157    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:29:34.640614 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630160    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:29:34.640614 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630163    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:29:34.640614 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630165    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:29:34.640614 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630168    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:29:34.640614 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630171    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:29:34.640614 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630175    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:29:34.640614 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630179    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:29:34.640614 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.630183    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:29:34.640614 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.630666    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:29:34.640614 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.639728    2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 19:29:34.640614 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.639745    2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 19:29:34.640614 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.640339    2572 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 19:29:34.640614 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.640347    2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640404    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640409    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640413    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640416    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640419    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640422    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640425    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640428    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640431    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640434    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640436    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640439    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640442    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640444    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640447    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640449    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640452    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640455    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640458    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:29:34.640960 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640462    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640465    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640468    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640470    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640473    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640476    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640479    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640482    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640484    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640487    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640490    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640493    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640496    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640498    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640501    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640503    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640506    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640508    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640511    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:29:34.641452 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640513    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640516    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640519    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640522    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640525    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640527    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640530    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640533    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640535    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640538    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640541    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640543    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640546    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640548    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640551    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640555    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640557    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640560    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640564    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:29:34.641913 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640568    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640573    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640576    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640579    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640582    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640584    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640587    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640590    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640592    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640595    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640597    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640600    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640603    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640606    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640608    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640611    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640613    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640616    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640618    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640621    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:29:34.642389 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640624    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640627    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640630    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640633    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640636    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640638    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640641    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640644    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640647    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640649    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640652    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640655    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640658    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640661    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640664    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640667    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640669    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640672    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640674    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640677    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:29:34.642895 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640680    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:29:34.643544 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640682    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:29:34.643544 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.640688    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:29:34.643544 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640802    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:29:34.643544 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640808    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:29:34.643544 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640811    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:29:34.643544 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640814    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:29:34.643544 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640817    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:29:34.643544 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640820    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:29:34.643544 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640823    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:29:34.643544 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640826    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:29:34.643544 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640828    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:29:34.643544 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640831    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:29:34.643544 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640834    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:29:34.643544 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640836    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:29:34.643544 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640839    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640842    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640845    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640847    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640850    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640853    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640855    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640858    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640860    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640863    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640866    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640869    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640871    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640874    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640876    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640879    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640882    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640885    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640887    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640890    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:29:34.643921 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640892    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:29:34.644431 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640896    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:29:34.644431 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640901    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:29:34.644431 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640904    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:29:34.644431 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640907    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:29:34.644431 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640909    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:29:34.644431 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640913    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:29:34.644431 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640916    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:29:34.644431 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640919    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:29:34.644431 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640923    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:29:34.644431 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640926    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:29:34.644431 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640929    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:29:34.644431 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640932    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:29:34.644431 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640936    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:29:34.644431 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640938    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:29:34.644431 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640941    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:29:34.644431 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640944    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:29:34.644431 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640946    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:29:34.644431 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640949    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640951    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640954    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640957    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640960    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640962    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640965    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640968    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640970    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640973    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640976    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640978    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640981    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640983    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640986    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640988    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640991    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640993    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640996    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.640999    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:29:34.644880 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641001    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641003    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641006    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641008    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641011    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641013    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641016    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641018    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641021    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641024    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641026    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641029    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641032    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641034    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641037    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641039    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641042    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641044    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641047    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641050    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:29:34.645371 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641052    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:29:34.645846 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641054    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:29:34.645846 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641057    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:29:34.645846 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641060    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:29:34.645846 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641062    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:29:34.645846 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641082    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:29:34.645846 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641086    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:29:34.645846 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641089    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:29:34.645846 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:34.641092    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:29:34.645846 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.641097    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:29:34.645846 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.641218    2572 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 19:29:34.645846 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.643166    2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 19:29:34.645846 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.644247    2572 server.go:1019] "Starting client certificate rotation"
Jul 28 19:29:34.645846 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.644353    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 19:29:34.646184 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.644381    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 19:29:34.646494 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.646478    2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 19:29:34.647980 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.647965    2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 19:29:34.656313 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.656297    2572 log.go:25] "Validated CRI v1 runtime API"
Jul 28 19:29:34.658329 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.658317    2572 log.go:25] "Validated CRI v1 image API"
Jul 28 19:29:34.659815 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.659790    2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 19:29:34.662038 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.662002    2572 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 861bc322-fdf2-46cf-8972-03a260574a89:/dev/nvme0n1p4 8e6df9d5-ef33-426a-ab0a-4a40c81b510d:/dev/nvme0n1p3]
Jul 28 19:29:34.662116 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.662037    2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 19:29:34.667983 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.667678    2572 manager.go:216] Machine: {Timestamp:2026-07-28 19:29:34.667210057 +0000 UTC m=+0.254872993 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095022 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2585e2abe75f88f98d45ec4738d1f0 SystemUUID:ec2585e2-abe7-5f88-f98d-45ec4738d1f0 BootID:d7a853c8-f6a1-442f-9335-05268e476ad6 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b3:56:b8:c5:f1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b3:56:b8:c5:f1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:bf:93:91:49:d4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 19:29:34.668482 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.668462    2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 19:29:34.668611 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.668592    2572 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 19:29:34.671325 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.671145    2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 19:29:34.671455 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.671328    2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-37.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 19:29:34.672034 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.672023    2572 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 19:29:34.672084 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.672036    2572 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 19:29:34.672084 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.672044    2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 19:29:34.672766 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.672751    2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 19:29:34.673520 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.673509    2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 19:29:34.675967 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.675953    2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 19:29:34.676775 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.676762    2572 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 19:29:34.676888 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.676879    2572 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 19:29:34.676926 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.676908    2572 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 19:29:34.676955 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.676936    2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 19:29:34.676955 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.676950    2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 19:29:34.677036 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.676959    2572 kubelet.go:398] "Adding apiserver pod source"
Jul 28 19:29:34.677036 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.676967    2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 19:29:34.677330 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.677305    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 19:29:34.677330 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.677325    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 19:29:34.679035 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.679016    2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 19:29:34.680737 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.680724    2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 19:29:34.680788 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.680755    2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 19:29:34.682207 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.682197    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 19:29:34.682239 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.682213    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 19:29:34.682239 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.682219    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 19:29:34.682239 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.682224    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 19:29:34.682239 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.682235    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 19:29:34.682432 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.682243    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 19:29:34.682432 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.682253    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 19:29:34.682432 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.682258    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 19:29:34.682432 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.682264    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 19:29:34.682432 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.682269    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 19:29:34.682432 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.682279    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 19:29:34.682432 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.682284    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 19:29:34.682866 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.682854    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 19:29:34.682866 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.682865    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 19:29:34.685964 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.685948    2572 server.go:1268] "Started kubelet"
Jul 28 19:29:34.686125 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.686030    2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 19:29:34.686184 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.686149    2572 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 19:29:34.686272 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.686059    2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 19:29:34.686869 ip-10-0-138-37 systemd[1]: Started Kubernetes Kubelet.
Jul 28 19:29:34.687762 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.687731    2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 19:29:34.688715 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.688697    2572 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 19:29:34.688926 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.688903    2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-37.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 19:29:34.689023 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.688980    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 19:29:34.689139 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.689118    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-37.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 19:29:34.690832 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.690812    2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 19:29:34.694152 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.693266    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-37.ec2.internal.18c68adacf63c154  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-37.ec2.internal,UID:ip-10-0-138-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-37.ec2.internal,},FirstTimestamp:2026-07-28 19:29:34.685905236 +0000 UTC m=+0.273568172,LastTimestamp:2026-07-28 19:29:34.685905236 +0000 UTC m=+0.273568172,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-37.ec2.internal,}"
Jul 28 19:29:34.695334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.695319    2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 19:29:34.695405 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.695332    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 19:29:34.695460 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.695444    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found"
Jul 28 19:29:34.695875 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.695858    2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 19:29:34.695949 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.695881    2572 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 19:29:34.695949 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.695896    2572 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 19:29:34.696093 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.696015    2572 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 19:29:34.696093 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.696027    2572 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 19:29:34.696176 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.696147    2572 factory.go:55] Registering systemd factory
Jul 28 19:29:34.696176 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.696159    2572 factory.go:223] Registration of the systemd container factory successfully
Jul 28 19:29:34.696378 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.696363    2572 factory.go:153] Registering CRI-O factory
Jul 28 19:29:34.696439 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.696381    2572 factory.go:223] Registration of the crio container factory successfully
Jul 28 19:29:34.696482 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.696449    2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 19:29:34.696482 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.696469    2572 factory.go:103] Registering Raw factory
Jul 28 19:29:34.696571 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.696501    2572 manager.go:1196] Started watching for new ooms in manager
Jul 28 19:29:34.696814 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.696775    2572 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-37.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 19:29:34.696925 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.696806    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 19:29:34.697321 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.697306    2572 manager.go:349] Starting recovery of all containers
Jul 28 19:29:34.708540 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.708519    2572 manager.go:354] Recovery completed
Jul 28 19:29:34.710196 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.710038    2572 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 28 19:29:34.713213 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.713135    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:29:34.715096 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.715079    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:29:34.715143 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.715107    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:29:34.715143 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.715119    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:29:34.715632 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.715611    2572 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 19:29:34.715632 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.715629    2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 19:29:34.715632 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.715633    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:29:34.715793 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.715650    2572 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 19:29:34.716294 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.716280    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:29:34.716346 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.716307    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:29:34.716346 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.716319    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:29:34.717172 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.717157    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rt4xg"
Jul 28 19:29:34.718222 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.718210    2572 policy_none.go:49] "None policy: Start"
Jul 28 19:29:34.718255 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.718227    2572 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 19:29:34.718255 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.718237    2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 19:29:34.720208 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.720197    2572 policy_none.go:47] "Start"
Jul 28 19:29:34.724163 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.724012    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-37.ec2.internal.18c68adad121265e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-37.ec2.internal,UID:ip-10-0-138-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-37.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-37.ec2.internal,},FirstTimestamp:2026-07-28 19:29:34.715094622 +0000 UTC m=+0.302757559,LastTimestamp:2026-07-28 19:29:34.715094622 +0000 UTC m=+0.302757559,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-37.ec2.internal,}"
Jul 28 19:29:34.724617 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.724599    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rt4xg"
Jul 28 19:29:34.726188 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.726125    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-37.ec2.internal.18c68adad1216bee  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-37.ec2.internal,UID:ip-10-0-138-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-37.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-37.ec2.internal,},FirstTimestamp:2026-07-28 19:29:34.71511243 +0000 UTC m=+0.302775367,LastTimestamp:2026-07-28 19:29:34.71511243 +0000 UTC m=+0.302775367,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-37.ec2.internal,}"
Jul 28 19:29:34.756696 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.756676    2572 manager.go:337] "Starting Device Plugin manager"
Jul 28 19:29:34.756777 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.756706    2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 19:29:34.756777 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.756715    2572 server.go:87] "Starting device plugin registration server"
Jul 28 19:29:34.756966 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.756956    2572 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 19:29:34.757010 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.756970    2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 19:29:34.757098 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.757064    2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 19:29:34.757182 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.757160    2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 19:29:34.757182 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.757173    2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 19:29:34.757688 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.757670    2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 19:29:34.757774 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.757721    2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-37.ec2.internal\" not found"
Jul 28 19:29:34.856338 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.856291    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 19:29:34.858433 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.857570    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 19:29:34.858433 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.857603    2572 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 19:29:34.858433 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.857626    2572 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 19:29:34.858433 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.857659    2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 19:29:34.858433 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.857802    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:29:34.858684 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.858609    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:29:34.858684 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.858638    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:29:34.858684 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.858652    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:29:34.858684 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.858683    2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-37.ec2.internal"
Jul 28 19:29:34.859502 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.859485    2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:29:34.867125 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.867105    2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-37.ec2.internal"
Jul 28 19:29:34.867203 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.867127    2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-37.ec2.internal\": node \"ip-10-0-138-37.ec2.internal\" not found"
Jul 28 19:29:34.892348 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.892300    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found"
Jul 28 19:29:34.958150 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.958110    2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal"]
Jul 28 19:29:34.958242 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.958213    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:29:34.960030 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.960013    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:29:34.960116 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.960048    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:29:34.960116 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.960063    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:29:34.962599 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.962586    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:29:34.962794 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.962767    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal"
Jul 28 19:29:34.962847 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.962819    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:29:34.963434 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.963418    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:29:34.963434 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.963428    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:29:34.963545 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.963446    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:29:34.963545 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.963452    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:29:34.963545 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.963459    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:29:34.963545 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.963463    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:29:34.965925 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.965911    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal"
Jul 28 19:29:34.965996 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.965944    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:29:34.966717 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.966703    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:29:34.966787 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.966736    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:29:34.966787 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.966764    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:29:34.991965 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.991942    2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-37.ec2.internal\" not found" node="ip-10-0-138-37.ec2.internal"
Jul 28 19:29:34.992972 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.992953    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found"
Jul 28 19:29:34.995543 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:34.995528    2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-37.ec2.internal\" not found" node="ip-10-0-138-37.ec2.internal"
Jul 28 19:29:34.997891 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.997873    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/70c009bc8352f8c83009ce9d3b8a04d2-config\") pod \"kube-apiserver-proxy-ip-10-0-138-37.ec2.internal\" (UID: \"70c009bc8352f8c83009ce9d3b8a04d2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal"
Jul 28 19:29:34.997967 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.997905    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7530272d554f2ab529547b87329e5402-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal\" (UID: \"7530272d554f2ab529547b87329e5402\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal"
Jul 28 19:29:34.997967 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:34.997931    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7530272d554f2ab529547b87329e5402-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal\" (UID: \"7530272d554f2ab529547b87329e5402\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal"
Jul 28 19:29:35.093625 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:35.093605    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found"
Jul 28 19:29:35.098237 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:35.098221    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/70c009bc8352f8c83009ce9d3b8a04d2-config\") pod \"kube-apiserver-proxy-ip-10-0-138-37.ec2.internal\" (UID: \"70c009bc8352f8c83009ce9d3b8a04d2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal"
Jul 28 19:29:35.098297 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:35.098252    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7530272d554f2ab529547b87329e5402-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal\" (UID: \"7530272d554f2ab529547b87329e5402\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal"
Jul 28 19:29:35.098297 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:35.098269    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7530272d554f2ab529547b87329e5402-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal\" (UID: \"7530272d554f2ab529547b87329e5402\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal"
Jul 28 19:29:35.098405 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:35.098317    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/70c009bc8352f8c83009ce9d3b8a04d2-config\") pod \"kube-apiserver-proxy-ip-10-0-138-37.ec2.internal\" (UID: \"70c009bc8352f8c83009ce9d3b8a04d2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal"
Jul 28 19:29:35.098405 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:35.098384    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7530272d554f2ab529547b87329e5402-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal\" (UID: \"7530272d554f2ab529547b87329e5402\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal"
Jul 28 19:29:35.098494 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:35.098394    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7530272d554f2ab529547b87329e5402-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal\" (UID: \"7530272d554f2ab529547b87329e5402\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal"
Jul 28 19:29:35.194884 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:35.194820    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found"
Jul 28 19:29:35.299285 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:35.299244    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found"
Jul 28 19:29:35.299285 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:35.299292    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal"
Jul 28 19:29:35.299422 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:35.299296    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal"
Jul 28 19:29:35.399801 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:35.399776    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found"
Jul 28 19:29:35.500157 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:35.500104    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found"
Jul 28 19:29:35.601109 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:35.601088    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found"
Jul 28 19:29:35.643624 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:35.643600    2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 19:29:35.644196 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:35.643738    2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 19:29:35.701485 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:35.701467    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found"
Jul 28 19:29:35.729643 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:35.729614    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 19:24:35 +0000 UTC" deadline="2028-05-03 21:48:13.357074276 +0000 UTC"
Jul 28 19:29:35.729643 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:35.729638    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15482h18m37.62743949s"
Jul 28 19:29:35.736124 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:35.736108    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:29:35.737487 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:35.737469    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:29:35.737601 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:35.737507    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:29:35.737601 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:35.737523    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:29:35.802346 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:35.802293    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found"
Jul 28 19:29:35.860921 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:35.860900    2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:29:35.862950 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:35.862912    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c009bc8352f8c83009ce9d3b8a04d2.slice/crio-045bd3ad54327d7ea2a656e60a1fd3a09d9b24fd730dae6c97b17c3f6dcece0f WatchSource:0}: Error finding container 045bd3ad54327d7ea2a656e60a1fd3a09d9b24fd730dae6c97b17c3f6dcece0f: Status 404 returned error can't find the container with id 045bd3ad54327d7ea2a656e60a1fd3a09d9b24fd730dae6c97b17c3f6dcece0f
Jul 28 19:29:35.868379 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:35.868361    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:29:35.873798 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:35.873773    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7530272d554f2ab529547b87329e5402.slice/crio-6c3cebbcabca5cf0822d7901e91078a7788b913c591102125c798943ab8ae10c WatchSource:0}: Error finding container 6c3cebbcabca5cf0822d7901e91078a7788b913c591102125c798943ab8ae10c: Status 404 returned error can't find the container with id 6c3cebbcabca5cf0822d7901e91078a7788b913c591102125c798943ab8ae10c
Jul 28 19:29:35.902507 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:35.902482    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found"
Jul 28 19:29:36.003529 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:36.003512    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found"
Jul 28 19:29:36.067805 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.067758    2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:29:36.096886 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.096861    2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal"
Jul 28 19:29:36.107288 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.107269    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 19:29:36.108161 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.108149    2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal"
Jul 28 19:29:36.120045 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.120026    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 19:29:36.219829 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.219811    2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:29:36.679346 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.679316    2572 apiserver.go:52] "Watching apiserver"
Jul 28 19:29:36.687801 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.687772    2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 19:29:36.688370 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.688344    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-5hfx5","kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf","openshift-cluster-node-tuning-operator/tuned-kqtsk","openshift-image-registry/node-ca-tgq2d","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal","openshift-multus/multus-cnk8n","openshift-dns/node-resolver-td8fx","openshift-multus/multus-additional-cni-plugins-bsh96","openshift-multus/network-metrics-daemon-8b8d6","openshift-network-diagnostics/network-check-target-cx4jl","openshift-network-operator/iptables-alerter-d8492","openshift-ovn-kubernetes/ovnkube-node-xf5lz","kube-system/global-pull-secret-syncer-vlfs6"]
Jul 28 19:29:36.691576 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.691554    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.694150 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.694126    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.694633 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.694588    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 19:29:36.694749 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.694727    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8fzhd\""
Jul 28 19:29:36.695284 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.695260    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 19:29:36.695375 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.695317    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 19:29:36.695439 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.695266    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 19:29:36.695589 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.695566    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 19:29:36.696689 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.696519    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 19:29:36.698051 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.697255    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jvhr5\""
Jul 28 19:29:36.698051 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.697462    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 19:29:36.698051 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.697679    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 19:29:36.698051 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.697892    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 19:29:36.698051 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.698016    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 19:29:36.698373 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.698267    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.699486 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.699463    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 19:29:36.700560 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.700294    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.700945 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.700923    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 19:29:36.701037 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.700942    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 19:29:36.701247 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.701231    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 19:29:36.701369 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.701350    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pxpfr\""
Jul 28 19:29:36.703078 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.702462    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:29:36.703378 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.703358    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nm4d9\""
Jul 28 19:29:36.703647 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.703632    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 19:29:36.704410 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.704385    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tgq2d"
Jul 28 19:29:36.706250 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.706185    2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 19:29:36.706480 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.706463    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:36.706605 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:36.706548    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlfs6" podUID="2c007a53-c536-4083-ab29-644788d631ba"
Jul 28 19:29:36.707091 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.707070    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6zlp5\""
Jul 28 19:29:36.707366 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.707326    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 19:29:36.707577 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.707558    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 19:29:36.707884 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.707863    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 19:29:36.708299 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708267    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a00376ce-82e8-47b8-917b-9ed30dd65272-multus-daemon-config\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.708383 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708302    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-cni-bin\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.708383 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708332    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-device-dir\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.708383 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708361    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.708541 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708392    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-multus-cni-dir\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.708541 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708418    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-cnibin\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.708541 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708451    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhp9h\" (UniqueName: \"kubernetes.io/projected/a00376ce-82e8-47b8-917b-9ed30dd65272-kube-api-access-fhp9h\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.708541 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708485    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-etc-openvswitch\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.708541 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708531    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/78af26d1-d3f7-4ef9-b8d5-803606097739-env-overrides\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.708792 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708558    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-host-run-k8s-cni-cncf-io\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.708792 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708583    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-run-systemd\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.708792 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708630    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-var-lib-openvswitch\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.708792 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708693    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-sys-fs\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.708792 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708746    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-os-release\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.709044 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708806    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-host-var-lib-cni-multus\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.709044 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708858    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-multus-conf-dir\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.709044 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708882    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-run-ovn\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.709044 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708886    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.709044 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708920    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.709044 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708952    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-socket-dir\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.709044 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.708997    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-registration-dir\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.709044 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709023    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-host-run-netns\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.709511 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709158    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-etc-kubernetes\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.709511 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709255    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-slash\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.709511 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709281    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-run-openvswitch\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.709511 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709309    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-run-ovn-kubernetes\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.709511 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709357    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6npk\" (UniqueName: \"kubernetes.io/projected/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-kube-api-access-p6npk\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.709511 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709393    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-system-cni-dir\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.709511 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709422    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a00376ce-82e8-47b8-917b-9ed30dd65272-cni-binary-copy\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.709511 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709452    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-kubelet\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.709912 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709675    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-log-socket\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.709912 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709715    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-cni-netd\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.709912 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709741    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/78af26d1-d3f7-4ef9-b8d5-803606097739-ovnkube-config\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.709912 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709768    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/78af26d1-d3f7-4ef9-b8d5-803606097739-ovn-node-metrics-cert\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.709912 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709792    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.709912 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709819    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-multus-socket-dir-parent\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.709912 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709849    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-host-var-lib-cni-bin\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.709912 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709891    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-host-var-lib-kubelet\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.710331 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709932    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-hostroot\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.710331 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.709985    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-host-run-multus-certs\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.710331 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.710012    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-systemd-units\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.710331 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.710043    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-run-netns\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.710331 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.710072    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-node-log\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.710331 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.710099    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/78af26d1-d3f7-4ef9-b8d5-803606097739-ovnkube-script-lib\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.710331 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.710176    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j8t6\" (UniqueName: \"kubernetes.io/projected/78af26d1-d3f7-4ef9-b8d5-803606097739-kube-api-access-4j8t6\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.716166 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.711675    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5hfx5"
Jul 28 19:29:36.716166 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.712299    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 19:29:36.716166 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.712611    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zxskh\""
Jul 28 19:29:36.716166 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.712985    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 19:29:36.716857 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.716836    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zljrt\""
Jul 28 19:29:36.717627 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.717608    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 19:29:36.722714 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.722691    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 19:29:36.724136 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.724117    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-td8fx"
Jul 28 19:29:36.726493 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.726281    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 19:29:36.726659 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.726642    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 19:29:36.726873 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.726854    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7kwwt\""
Jul 28 19:29:36.727090 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.727072    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:36.727217 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:36.727191    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:29:36.729319 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.729223    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:36.729419 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:36.729388    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cx4jl" podUID="8965338e-4b6f-4d05-905f-15d353c0ea7f"
Jul 28 19:29:36.730664 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.730635    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pzcts"
Jul 28 19:29:36.731603 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.731584    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d8492"
Jul 28 19:29:36.734635 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.734612    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 19:29:36.734852 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.734831    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lskxq\""
Jul 28 19:29:36.735178 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.735152    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:29:36.735691 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.735661    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 19:29:36.738527 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.738506    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pzcts"
Jul 28 19:29:36.797904 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.797885    2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 19:29:36.810528 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810503    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-systemd-units\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.810639 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810543    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-run-netns\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.810639 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810570    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-node-log\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.810639 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810605    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj7lw\" (UniqueName: \"kubernetes.io/projected/303f736b-24a3-4e9b-8904-e4fde37ab7e0-kube-api-access-mj7lw\") pod \"network-metrics-daemon-8b8d6\" (UID: \"303f736b-24a3-4e9b-8904-e4fde37ab7e0\") " pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:36.810639 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810630    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-sysctl-d\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.810849 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810647    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-sys\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.810849 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810663    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs\") pod \"network-metrics-daemon-8b8d6\" (UID: \"303f736b-24a3-4e9b-8904-e4fde37ab7e0\") " pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:36.810849 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810680    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5-hosts-file\") pod \"node-resolver-td8fx\" (UID: \"4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5\") " pod="openshift-dns/node-resolver-td8fx"
Jul 28 19:29:36.810849 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810697    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a00376ce-82e8-47b8-917b-9ed30dd65272-multus-daemon-config\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.810849 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810711    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/be4dd071-8c56-455a-bb1e-440b5b292f3b-serviceca\") pod \"node-ca-tgq2d\" (UID: \"be4dd071-8c56-455a-bb1e-440b5b292f3b\") " pod="openshift-image-registry/node-ca-tgq2d"
Jul 28 19:29:36.810849 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810727    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/babffeed-a4c6-47f8-b0a6-eee191647b19-iptables-alerter-script\") pod \"iptables-alerter-d8492\" (UID: \"babffeed-a4c6-47f8-b0a6-eee191647b19\") " pod="openshift-network-operator/iptables-alerter-d8492"
Jul 28 19:29:36.810849 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810747    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/babffeed-a4c6-47f8-b0a6-eee191647b19-host-slash\") pod \"iptables-alerter-d8492\" (UID: \"babffeed-a4c6-47f8-b0a6-eee191647b19\") " pod="openshift-network-operator/iptables-alerter-d8492"
Jul 28 19:29:36.810849 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810762    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-tuned\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.810849 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810776    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6skpj\" (UniqueName: \"kubernetes.io/projected/def32d52-4c44-402a-a3e3-354a1ce6ce1d-kube-api-access-6skpj\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.810849 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810794    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fdf14d1f-4a97-49ed-a0f4-f28fd90e5482-konnectivity-ca\") pod \"konnectivity-agent-5hfx5\" (UID: \"fdf14d1f-4a97-49ed-a0f4-f28fd90e5482\") " pod="kube-system/konnectivity-agent-5hfx5"
Jul 28 19:29:36.810849 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810822    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-etc-openvswitch\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.811464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810874    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/78af26d1-d3f7-4ef9-b8d5-803606097739-env-overrides\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.811464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810893    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-systemd-units\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.811464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810923    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-run\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.811464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810951    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2acc75c7-ac70-4ca4-b588-dde7608024c3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.811464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810984    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5-tmp-dir\") pod \"node-resolver-td8fx\" (UID: \"4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5\") " pod="openshift-dns/node-resolver-td8fx"
Jul 28 19:29:36.811464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810992    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-node-log\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.811464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811010    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-host-run-k8s-cni-cncf-io\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.811464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811036    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-run-systemd\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.811464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811063    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-var-lib-openvswitch\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.811464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811104    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-sys-fs\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.811464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811156    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-host-run-k8s-cni-cncf-io\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.811464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811170    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be4dd071-8c56-455a-bb1e-440b5b292f3b-host\") pod \"node-ca-tgq2d\" (UID: \"be4dd071-8c56-455a-bb1e-440b5b292f3b\") " pod="openshift-image-registry/node-ca-tgq2d"
Jul 28 19:29:36.811464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.810956    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-run-netns\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.811464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811212    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-etc-openvswitch\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.811464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811211    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2acc75c7-ac70-4ca4-b588-dde7608024c3-cnibin\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.811464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811260    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flv9j\" (UniqueName: \"kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j\") pod \"network-check-target-cx4jl\" (UID: \"8965338e-4b6f-4d05-905f-15d353c0ea7f\") " pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:36.811464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811273    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-var-lib-openvswitch\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.812334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811296    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-os-release\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.812334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811305    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-sys-fs\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.812334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811324    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-run-systemd\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.812334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811322    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-host-var-lib-cni-multus\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.812334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811376    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-multus-conf-dir\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.812334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811403    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-run-ovn\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.812334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811417    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-os-release\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.812334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811427    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-socket-dir\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.812334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811427    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-multus-conf-dir\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.812334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811464    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-host-var-lib-cni-multus\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.812334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811467    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-registration-dir\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.812334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811471    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-run-ovn\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.812334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811520    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-registration-dir\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.812334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811549    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-sysconfig\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.812334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811568    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-socket-dir\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.812334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811581    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-host-run-netns\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.812334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811623    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-run-openvswitch\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.812334 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811599    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/78af26d1-d3f7-4ef9-b8d5-803606097739-env-overrides\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.813058 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811648    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6npk\" (UniqueName: \"kubernetes.io/projected/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-kube-api-access-p6npk\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.813058 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811643    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-host-run-netns\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.813058 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811675    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-cni-netd\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.813058 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811679    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a00376ce-82e8-47b8-917b-9ed30dd65272-multus-daemon-config\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.813058 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811691    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-host\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.813058 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811709    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-cni-netd\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.813058 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811720    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2acc75c7-ac70-4ca4-b588-dde7608024c3-system-cni-dir\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.813058 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811750    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-host-var-lib-kubelet\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.813058 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811683    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-run-openvswitch\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.813058 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811776    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-hostroot\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.813058 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811801    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-host-run-multus-certs\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.813058 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811811    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-host-var-lib-kubelet\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.813058 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811828    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j8t6\" (UniqueName: \"kubernetes.io/projected/78af26d1-d3f7-4ef9-b8d5-803606097739-kube-api-access-4j8t6\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.813058 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811845    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-host-run-multus-certs\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.813058 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811850    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-hostroot\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.813058 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811862    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz2pn\" (UniqueName: \"kubernetes.io/projected/babffeed-a4c6-47f8-b0a6-eee191647b19-kube-api-access-rz2pn\") pod \"iptables-alerter-d8492\" (UID: \"babffeed-a4c6-47f8-b0a6-eee191647b19\") " pod="openshift-network-operator/iptables-alerter-d8492"
Jul 28 19:29:36.813058 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811886    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-lib-modules\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.813931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811913    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2acc75c7-ac70-4ca4-b588-dde7608024c3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.813931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811939    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/78af26d1-d3f7-4ef9-b8d5-803606097739-ovnkube-script-lib\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.813931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811964    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2c007a53-c536-4083-ab29-644788d631ba-kubelet-config\") pod \"global-pull-secret-syncer-vlfs6\" (UID: \"2c007a53-c536-4083-ab29-644788d631ba\") " pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:36.813931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.811988    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-kubernetes\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.813931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812012    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-var-lib-kubelet\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.813931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812037    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/def32d52-4c44-402a-a3e3-354a1ce6ce1d-tmp\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.813931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812071    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2acc75c7-ac70-4ca4-b588-dde7608024c3-cni-binary-copy\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.813931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812099    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-cni-bin\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.813931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812147    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-device-dir\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.813931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812174    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-456w7\" (UniqueName: \"kubernetes.io/projected/be4dd071-8c56-455a-bb1e-440b5b292f3b-kube-api-access-456w7\") pod \"node-ca-tgq2d\" (UID: \"be4dd071-8c56-455a-bb1e-440b5b292f3b\") " pod="openshift-image-registry/node-ca-tgq2d"
Jul 28 19:29:36.813931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812185    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-cni-bin\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.813931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812196    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fdf14d1f-4a97-49ed-a0f4-f28fd90e5482-agent-certs\") pod \"konnectivity-agent-5hfx5\" (UID: \"fdf14d1f-4a97-49ed-a0f4-f28fd90e5482\") " pod="kube-system/konnectivity-agent-5hfx5"
Jul 28 19:29:36.813931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812224    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-multus-cni-dir\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.813931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812234    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-device-dir\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.813931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812264    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-cnibin\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.813931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812336    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-multus-cni-dir\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.813931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812339    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhp9h\" (UniqueName: \"kubernetes.io/projected/a00376ce-82e8-47b8-917b-9ed30dd65272-kube-api-access-fhp9h\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.814704 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812358    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-cnibin\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.814704 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812374    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2c007a53-c536-4083-ab29-644788d631ba-dbus\") pod \"global-pull-secret-syncer-vlfs6\" (UID: \"2c007a53-c536-4083-ab29-644788d631ba\") " pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:36.814704 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812403    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-systemd\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.814704 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812426    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2acc75c7-ac70-4ca4-b588-dde7608024c3-os-release\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.814704 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812468    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skh86\" (UniqueName: \"kubernetes.io/projected/2acc75c7-ac70-4ca4-b588-dde7608024c3-kube-api-access-skh86\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.814704 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812496    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-modprobe-d\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.814704 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812494    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/78af26d1-d3f7-4ef9-b8d5-803606097739-ovnkube-script-lib\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.814704 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812531    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-sysctl-conf\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.814704 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812581    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2acc75c7-ac70-4ca4-b588-dde7608024c3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.814704 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812615    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.814704 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812643    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret\") pod \"global-pull-secret-syncer-vlfs6\" (UID: \"2c007a53-c536-4083-ab29-644788d631ba\") " pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:36.814704 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812668    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-etc-kubernetes\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.814704 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812692    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-slash\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.814704 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812717    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.814704 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812717    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-run-ovn-kubernetes\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.814704 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812764    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-run-ovn-kubernetes\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.815445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812763    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qxdl\" (UniqueName: \"kubernetes.io/projected/4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5-kube-api-access-2qxdl\") pod \"node-resolver-td8fx\" (UID: \"4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5\") " pod="openshift-dns/node-resolver-td8fx"
Jul 28 19:29:36.815445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812795    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-etc-kubernetes\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.815445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812802    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-system-cni-dir\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.815445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812826    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-slash\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.815445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812830    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a00376ce-82e8-47b8-917b-9ed30dd65272-cni-binary-copy\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.815445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812854    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-kubelet\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.815445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812863    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-system-cni-dir\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.815445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812877    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-log-socket\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.815445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812901    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/78af26d1-d3f7-4ef9-b8d5-803606097739-ovnkube-config\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.815445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812941    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/78af26d1-d3f7-4ef9-b8d5-803606097739-ovn-node-metrics-cert\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.815445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812966    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.815445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.812990    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.815445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.813015    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-multus-socket-dir-parent\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.815445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.813040    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-host-var-lib-cni-bin\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.815445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.813138    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-host-var-lib-cni-bin\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.815445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.813182    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-host-kubelet\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.815445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.813240    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.815445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.813250    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/78af26d1-d3f7-4ef9-b8d5-803606097739-log-socket\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.816306 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.813267    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a00376ce-82e8-47b8-917b-9ed30dd65272-cni-binary-copy\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.816306 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.813325    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a00376ce-82e8-47b8-917b-9ed30dd65272-multus-socket-dir-parent\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.816306 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.813341    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.816306 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.813615    2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 19:29:36.816306 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.813717    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/78af26d1-d3f7-4ef9-b8d5-803606097739-ovnkube-config\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.817230 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.817205    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/78af26d1-d3f7-4ef9-b8d5-803606097739-ovn-node-metrics-cert\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.821116 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.821052    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6npk\" (UniqueName: \"kubernetes.io/projected/112bf4e8-7d1b-4aa1-aba7-6970554df9ab-kube-api-access-p6npk\") pod \"aws-ebs-csi-driver-node-qzcrf\" (UID: \"112bf4e8-7d1b-4aa1-aba7-6970554df9ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:36.821734 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.821639    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j8t6\" (UniqueName: \"kubernetes.io/projected/78af26d1-d3f7-4ef9-b8d5-803606097739-kube-api-access-4j8t6\") pod \"ovnkube-node-xf5lz\" (UID: \"78af26d1-d3f7-4ef9-b8d5-803606097739\") " pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:36.821831 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.821741    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhp9h\" (UniqueName: \"kubernetes.io/projected/a00376ce-82e8-47b8-917b-9ed30dd65272-kube-api-access-fhp9h\") pod \"multus-cnk8n\" (UID: \"a00376ce-82e8-47b8-917b-9ed30dd65272\") " pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:36.863197 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.863147    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal" event={"ID":"7530272d554f2ab529547b87329e5402","Type":"ContainerStarted","Data":"6c3cebbcabca5cf0822d7901e91078a7788b913c591102125c798943ab8ae10c"}
Jul 28 19:29:36.864932 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.864903    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal" event={"ID":"70c009bc8352f8c83009ce9d3b8a04d2","Type":"ContainerStarted","Data":"045bd3ad54327d7ea2a656e60a1fd3a09d9b24fd730dae6c97b17c3f6dcece0f"}
Jul 28 19:29:36.914053 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914023    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj7lw\" (UniqueName: \"kubernetes.io/projected/303f736b-24a3-4e9b-8904-e4fde37ab7e0-kube-api-access-mj7lw\") pod \"network-metrics-daemon-8b8d6\" (UID: \"303f736b-24a3-4e9b-8904-e4fde37ab7e0\") " pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:36.914191 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914066    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-sysctl-d\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.914191 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914096    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-sys\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.914191 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914136    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs\") pod \"network-metrics-daemon-8b8d6\" (UID: \"303f736b-24a3-4e9b-8904-e4fde37ab7e0\") " pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:36.914191 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914161    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5-hosts-file\") pod \"node-resolver-td8fx\" (UID: \"4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5\") " pod="openshift-dns/node-resolver-td8fx"
Jul 28 19:29:36.914191 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914185    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/be4dd071-8c56-455a-bb1e-440b5b292f3b-serviceca\") pod \"node-ca-tgq2d\" (UID: \"be4dd071-8c56-455a-bb1e-440b5b292f3b\") " pod="openshift-image-registry/node-ca-tgq2d"
Jul 28 19:29:36.914508 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914211    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/babffeed-a4c6-47f8-b0a6-eee191647b19-iptables-alerter-script\") pod \"iptables-alerter-d8492\" (UID: \"babffeed-a4c6-47f8-b0a6-eee191647b19\") " pod="openshift-network-operator/iptables-alerter-d8492"
Jul 28 19:29:36.914508 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914236    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/babffeed-a4c6-47f8-b0a6-eee191647b19-host-slash\") pod \"iptables-alerter-d8492\" (UID: \"babffeed-a4c6-47f8-b0a6-eee191647b19\") " pod="openshift-network-operator/iptables-alerter-d8492"
Jul 28 19:29:36.914508 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914260    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-tuned\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.914508 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914258    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-sysctl-d\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.914508 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914284    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6skpj\" (UniqueName: \"kubernetes.io/projected/def32d52-4c44-402a-a3e3-354a1ce6ce1d-kube-api-access-6skpj\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.914508 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914309    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fdf14d1f-4a97-49ed-a0f4-f28fd90e5482-konnectivity-ca\") pod \"konnectivity-agent-5hfx5\" (UID: \"fdf14d1f-4a97-49ed-a0f4-f28fd90e5482\") " pod="kube-system/konnectivity-agent-5hfx5"
Jul 28 19:29:36.914508 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914334    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-run\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.914508 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914358    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2acc75c7-ac70-4ca4-b588-dde7608024c3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.914508 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914368    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-sys\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.914508 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914387    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5-tmp-dir\") pod \"node-resolver-td8fx\" (UID: \"4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5\") " pod="openshift-dns/node-resolver-td8fx"
Jul 28 19:29:36.914508 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914414    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be4dd071-8c56-455a-bb1e-440b5b292f3b-host\") pod \"node-ca-tgq2d\" (UID: \"be4dd071-8c56-455a-bb1e-440b5b292f3b\") " pod="openshift-image-registry/node-ca-tgq2d"
Jul 28 19:29:36.914508 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914437    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2acc75c7-ac70-4ca4-b588-dde7608024c3-cnibin\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.914508 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914464    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flv9j\" (UniqueName: \"kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j\") pod \"network-check-target-cx4jl\" (UID: \"8965338e-4b6f-4d05-905f-15d353c0ea7f\") " pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:36.914508 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:36.914474    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:29:36.914508 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914493    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-sysconfig\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.915287 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914521    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-host\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.915287 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:36.914546    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs podName:303f736b-24a3-4e9b-8904-e4fde37ab7e0 nodeName:}" failed. No retries permitted until 2026-07-28 19:29:37.414521154 +0000 UTC m=+3.002184126 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs") pod "network-metrics-daemon-8b8d6" (UID: "303f736b-24a3-4e9b-8904-e4fde37ab7e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:29:36.915287 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914582    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2acc75c7-ac70-4ca4-b588-dde7608024c3-system-cni-dir\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.915287 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914586    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-host\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.915287 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914620    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz2pn\" (UniqueName: \"kubernetes.io/projected/babffeed-a4c6-47f8-b0a6-eee191647b19-kube-api-access-rz2pn\") pod \"iptables-alerter-d8492\" (UID: \"babffeed-a4c6-47f8-b0a6-eee191647b19\") " pod="openshift-network-operator/iptables-alerter-d8492"
Jul 28 19:29:36.915287 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914647    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-lib-modules\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.915287 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914675    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2acc75c7-ac70-4ca4-b588-dde7608024c3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.915287 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914703    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2c007a53-c536-4083-ab29-644788d631ba-kubelet-config\") pod \"global-pull-secret-syncer-vlfs6\" (UID: \"2c007a53-c536-4083-ab29-644788d631ba\") " pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:36.915287 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914728    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-kubernetes\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.915287 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914753    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-var-lib-kubelet\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.915287 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914776    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/def32d52-4c44-402a-a3e3-354a1ce6ce1d-tmp\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.915287 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914802    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2acc75c7-ac70-4ca4-b588-dde7608024c3-cni-binary-copy\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.915287 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914873    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-456w7\" (UniqueName: \"kubernetes.io/projected/be4dd071-8c56-455a-bb1e-440b5b292f3b-kube-api-access-456w7\") pod \"node-ca-tgq2d\" (UID: \"be4dd071-8c56-455a-bb1e-440b5b292f3b\") " pod="openshift-image-registry/node-ca-tgq2d"
Jul 28 19:29:36.915287 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914901    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fdf14d1f-4a97-49ed-a0f4-f28fd90e5482-agent-certs\") pod \"konnectivity-agent-5hfx5\" (UID: \"fdf14d1f-4a97-49ed-a0f4-f28fd90e5482\") " pod="kube-system/konnectivity-agent-5hfx5"
Jul 28 19:29:36.915287 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914931    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2c007a53-c536-4083-ab29-644788d631ba-dbus\") pod \"global-pull-secret-syncer-vlfs6\" (UID: \"2c007a53-c536-4083-ab29-644788d631ba\") " pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:36.915287 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914957    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-systemd\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.916099 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.914981    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2acc75c7-ac70-4ca4-b588-dde7608024c3-os-release\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.916099 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.915008    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skh86\" (UniqueName: \"kubernetes.io/projected/2acc75c7-ac70-4ca4-b588-dde7608024c3-kube-api-access-skh86\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.916099 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.915038    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-modprobe-d\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.916099 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.915062    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-sysctl-conf\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.916099 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.915118    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2acc75c7-ac70-4ca4-b588-dde7608024c3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.916099 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.915138    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/babffeed-a4c6-47f8-b0a6-eee191647b19-iptables-alerter-script\") pod \"iptables-alerter-d8492\" (UID: \"babffeed-a4c6-47f8-b0a6-eee191647b19\") " pod="openshift-network-operator/iptables-alerter-d8492"
Jul 28 19:29:36.916099 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.915171    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret\") pod \"global-pull-secret-syncer-vlfs6\" (UID: \"2c007a53-c536-4083-ab29-644788d631ba\") " pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:36.916099 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.915224    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/babffeed-a4c6-47f8-b0a6-eee191647b19-host-slash\") pod \"iptables-alerter-d8492\" (UID: \"babffeed-a4c6-47f8-b0a6-eee191647b19\") " pod="openshift-network-operator/iptables-alerter-d8492"
Jul 28 19:29:36.916099 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.915250    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5-hosts-file\") pod \"node-resolver-td8fx\" (UID: \"4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5\") " pod="openshift-dns/node-resolver-td8fx"
Jul 28 19:29:36.916099 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.915259    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qxdl\" (UniqueName: \"kubernetes.io/projected/4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5-kube-api-access-2qxdl\") pod \"node-resolver-td8fx\" (UID: \"4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5\") " pod="openshift-dns/node-resolver-td8fx"
Jul 28 19:29:36.916099 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.915475    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2acc75c7-ac70-4ca4-b588-dde7608024c3-system-cni-dir\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.916099 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.915780    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-lib-modules\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.916099 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.915936    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2acc75c7-ac70-4ca4-b588-dde7608024c3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.916099 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.915993    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2c007a53-c536-4083-ab29-644788d631ba-kubelet-config\") pod \"global-pull-secret-syncer-vlfs6\" (UID: \"2c007a53-c536-4083-ab29-644788d631ba\") " pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:36.916099 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.916040    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-kubernetes\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.916099 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.916089    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-var-lib-kubelet\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.917009 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.916125    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2acc75c7-ac70-4ca4-b588-dde7608024c3-cnibin\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.917009 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.916227    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be4dd071-8c56-455a-bb1e-440b5b292f3b-host\") pod \"node-ca-tgq2d\" (UID: \"be4dd071-8c56-455a-bb1e-440b5b292f3b\") " pod="openshift-image-registry/node-ca-tgq2d"
Jul 28 19:29:36.917009 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.916306    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2acc75c7-ac70-4ca4-b588-dde7608024c3-os-release\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.917009 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.916336    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/be4dd071-8c56-455a-bb1e-440b5b292f3b-serviceca\") pod \"node-ca-tgq2d\" (UID: \"be4dd071-8c56-455a-bb1e-440b5b292f3b\") " pod="openshift-image-registry/node-ca-tgq2d"
Jul 28 19:29:36.917009 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.916413    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-run\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.917009 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.916457    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fdf14d1f-4a97-49ed-a0f4-f28fd90e5482-konnectivity-ca\") pod \"konnectivity-agent-5hfx5\" (UID: \"fdf14d1f-4a97-49ed-a0f4-f28fd90e5482\") " pod="kube-system/konnectivity-agent-5hfx5"
Jul 28 19:29:36.917009 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.916465    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5-tmp-dir\") pod \"node-resolver-td8fx\" (UID: \"4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5\") " pod="openshift-dns/node-resolver-td8fx"
Jul 28 19:29:36.917009 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.916583    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2acc75c7-ac70-4ca4-b588-dde7608024c3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.917009 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.916592    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2c007a53-c536-4083-ab29-644788d631ba-dbus\") pod \"global-pull-secret-syncer-vlfs6\" (UID: \"2c007a53-c536-4083-ab29-644788d631ba\") " pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:36.917009 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.916638    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-sysconfig\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.917009 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.916644    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-sysctl-conf\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.917009 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.916702    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-systemd\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.917009 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.916830    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-modprobe-d\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.917009 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.916875    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2acc75c7-ac70-4ca4-b588-dde7608024c3-cni-binary-copy\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.917009 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:36.916886    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:29:36.917009 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:36.916947    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret podName:2c007a53-c536-4083-ab29-644788d631ba nodeName:}" failed. No retries permitted until 2026-07-28 19:29:37.41692712 +0000 UTC m=+3.004590050 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret") pod "global-pull-secret-syncer-vlfs6" (UID: "2c007a53-c536-4083-ab29-644788d631ba") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:29:36.917837 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.917233    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2acc75c7-ac70-4ca4-b588-dde7608024c3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:36.919182 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.919160    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/def32d52-4c44-402a-a3e3-354a1ce6ce1d-tmp\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.919590 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.919546    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/def32d52-4c44-402a-a3e3-354a1ce6ce1d-etc-tuned\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.919826 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.919795    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fdf14d1f-4a97-49ed-a0f4-f28fd90e5482-agent-certs\") pod \"konnectivity-agent-5hfx5\" (UID: \"fdf14d1f-4a97-49ed-a0f4-f28fd90e5482\") " pod="kube-system/konnectivity-agent-5hfx5"
Jul 28 19:29:36.926592 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:36.926237    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:29:36.926592 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:36.926260    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:29:36.926592 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:36.926274    2572 projected.go:196] Error preparing data for projected volume kube-api-access-flv9j for pod openshift-network-diagnostics/network-check-target-cx4jl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:29:36.926592 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:36.926332    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j podName:8965338e-4b6f-4d05-905f-15d353c0ea7f nodeName:}" failed. No retries permitted until 2026-07-28 19:29:37.426315257 +0000 UTC m=+3.013978201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-flv9j" (UniqueName: "kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j") pod "network-check-target-cx4jl" (UID: "8965338e-4b6f-4d05-905f-15d353c0ea7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:29:36.928317 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.928257    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj7lw\" (UniqueName: \"kubernetes.io/projected/303f736b-24a3-4e9b-8904-e4fde37ab7e0-kube-api-access-mj7lw\") pod \"network-metrics-daemon-8b8d6\" (UID: \"303f736b-24a3-4e9b-8904-e4fde37ab7e0\") " pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:36.928492 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.928467    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6skpj\" (UniqueName: \"kubernetes.io/projected/def32d52-4c44-402a-a3e3-354a1ce6ce1d-kube-api-access-6skpj\") pod \"tuned-kqtsk\" (UID: \"def32d52-4c44-402a-a3e3-354a1ce6ce1d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:36.929194 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.929077    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz2pn\" (UniqueName: \"kubernetes.io/projected/babffeed-a4c6-47f8-b0a6-eee191647b19-kube-api-access-rz2pn\") pod \"iptables-alerter-d8492\" (UID: \"babffeed-a4c6-47f8-b0a6-eee191647b19\") " pod="openshift-network-operator/iptables-alerter-d8492"
Jul 28 19:29:36.929194 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.929159    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qxdl\" (UniqueName: \"kubernetes.io/projected/4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5-kube-api-access-2qxdl\") pod \"node-resolver-td8fx\" (UID: \"4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5\") " pod="openshift-dns/node-resolver-td8fx"
Jul 28 19:29:36.930611 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.930561    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-456w7\" (UniqueName: \"kubernetes.io/projected/be4dd071-8c56-455a-bb1e-440b5b292f3b-kube-api-access-456w7\") pod \"node-ca-tgq2d\" (UID: \"be4dd071-8c56-455a-bb1e-440b5b292f3b\") " pod="openshift-image-registry/node-ca-tgq2d"
Jul 28 19:29:36.930708 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:36.930652    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skh86\" (UniqueName: \"kubernetes.io/projected/2acc75c7-ac70-4ca4-b588-dde7608024c3-kube-api-access-skh86\") pod \"multus-additional-cni-plugins-bsh96\" (UID: \"2acc75c7-ac70-4ca4-b588-dde7608024c3\") " pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:37.008461 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.008433    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cnk8n"
Jul 28 19:29:37.022123 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.022093    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:29:37.039714 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.039689    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf"
Jul 28 19:29:37.047948 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.047926    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kqtsk"
Jul 28 19:29:37.057071 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.056922    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tgq2d"
Jul 28 19:29:37.064930 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.064911    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bsh96"
Jul 28 19:29:37.073845 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.073822    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5hfx5"
Jul 28 19:29:37.082714 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.082692    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-td8fx"
Jul 28 19:29:37.090542 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.090522    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d8492"
Jul 28 19:29:37.091025 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:37.090997    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2acc75c7_ac70_4ca4_b588_dde7608024c3.slice/crio-fa46e7001fb26cd4cf7d4ba23e1358db5aeced2f243036b9cac6727826970cb5 WatchSource:0}: Error finding container fa46e7001fb26cd4cf7d4ba23e1358db5aeced2f243036b9cac6727826970cb5: Status 404 returned error can't find the container with id fa46e7001fb26cd4cf7d4ba23e1358db5aeced2f243036b9cac6727826970cb5
Jul 28 19:29:37.091813 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:37.091691    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod112bf4e8_7d1b_4aa1_aba7_6970554df9ab.slice/crio-2cc608c1b58d6e4f421430bc611ec91bd6776fd00bc2c7216ec047590fc0fa0f WatchSource:0}: Error finding container 2cc608c1b58d6e4f421430bc611ec91bd6776fd00bc2c7216ec047590fc0fa0f: Status 404 returned error can't find the container with id 2cc608c1b58d6e4f421430bc611ec91bd6776fd00bc2c7216ec047590fc0fa0f
Jul 28 19:29:37.094975 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:37.094952    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda00376ce_82e8_47b8_917b_9ed30dd65272.slice/crio-24dcca221fd43979f268555b9f1077f77b770304160c5d271fa6a790961181a2 WatchSource:0}: Error finding container 24dcca221fd43979f268555b9f1077f77b770304160c5d271fa6a790961181a2: Status 404 returned error can't find the container with id 24dcca221fd43979f268555b9f1077f77b770304160c5d271fa6a790961181a2
Jul 28 19:29:37.097549 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:37.097442    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddef32d52_4c44_402a_a3e3_354a1ce6ce1d.slice/crio-1cec1c7e151a77569c9f68d779b9b74f135c71162940f46dd085a85bbc888197 WatchSource:0}: Error finding container 1cec1c7e151a77569c9f68d779b9b74f135c71162940f46dd085a85bbc888197: Status 404 returned error can't find the container with id 1cec1c7e151a77569c9f68d779b9b74f135c71162940f46dd085a85bbc888197
Jul 28 19:29:37.098304 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:37.098278    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe4dd071_8c56_455a_bb1e_440b5b292f3b.slice/crio-0db478b894f2a6f6030691a7b7994a2433ec607aebc487801ecbfce23939e891 WatchSource:0}: Error finding container 0db478b894f2a6f6030691a7b7994a2433ec607aebc487801ecbfce23939e891: Status 404 returned error can't find the container with id 0db478b894f2a6f6030691a7b7994a2433ec607aebc487801ecbfce23939e891
Jul 28 19:29:37.124261 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:37.124228    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdf14d1f_4a97_49ed_a0f4_f28fd90e5482.slice/crio-e142b2eeec9e5933854e5c67ac11e6e9af6803622d0aebe36d994a7b6348e6de WatchSource:0}: Error finding container e142b2eeec9e5933854e5c67ac11e6e9af6803622d0aebe36d994a7b6348e6de: Status 404 returned error can't find the container with id e142b2eeec9e5933854e5c67ac11e6e9af6803622d0aebe36d994a7b6348e6de
Jul 28 19:29:37.243282 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.243211    2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:29:37.419594 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.419555    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs\") pod \"network-metrics-daemon-8b8d6\" (UID: \"303f736b-24a3-4e9b-8904-e4fde37ab7e0\") " pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:37.419782 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.419640    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret\") pod \"global-pull-secret-syncer-vlfs6\" (UID: \"2c007a53-c536-4083-ab29-644788d631ba\") " pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:37.419782 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:37.419757    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:29:37.419886 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:37.419841    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret podName:2c007a53-c536-4083-ab29-644788d631ba nodeName:}" failed. No retries permitted until 2026-07-28 19:29:38.419820086 +0000 UTC m=+4.007483016 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret") pod "global-pull-secret-syncer-vlfs6" (UID: "2c007a53-c536-4083-ab29-644788d631ba") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:29:37.419886 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:37.419757    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:29:37.419978 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:37.419921    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs podName:303f736b-24a3-4e9b-8904-e4fde37ab7e0 nodeName:}" failed. No retries permitted until 2026-07-28 19:29:38.419903822 +0000 UTC m=+4.007566753 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs") pod "network-metrics-daemon-8b8d6" (UID: "303f736b-24a3-4e9b-8904-e4fde37ab7e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:29:37.520765 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.520688    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flv9j\" (UniqueName: \"kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j\") pod \"network-check-target-cx4jl\" (UID: \"8965338e-4b6f-4d05-905f-15d353c0ea7f\") " pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:37.520916 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:37.520862    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:29:37.520916 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:37.520889    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:29:37.520916 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:37.520899    2572 projected.go:196] Error preparing data for projected volume kube-api-access-flv9j for pod openshift-network-diagnostics/network-check-target-cx4jl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:29:37.521031 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:37.520953    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j podName:8965338e-4b6f-4d05-905f-15d353c0ea7f nodeName:}" failed. No retries permitted until 2026-07-28 19:29:38.520939146 +0000 UTC m=+4.108602073 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-flv9j" (UniqueName: "kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j") pod "network-check-target-cx4jl" (UID: "8965338e-4b6f-4d05-905f-15d353c0ea7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:29:37.742209 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.742139    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 19:24:36 +0000 UTC" deadline="2028-01-24 07:25:05.49367464 +0000 UTC"
Jul 28 19:29:37.742209 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.742183    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13067h55m27.751497199s"
Jul 28 19:29:37.820699 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:29:37.820660    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbabffeed_a4c6_47f8_b0a6_eee191647b19.slice/crio-cfe6c8d2b6d12db2090c6ccebd6def508fe1cc28b5692d0eca2fb87cb3b105c4 WatchSource:0}: Error finding container cfe6c8d2b6d12db2090c6ccebd6def508fe1cc28b5692d0eca2fb87cb3b105c4: Status 404 returned error can't find the container with id cfe6c8d2b6d12db2090c6ccebd6def508fe1cc28b5692d0eca2fb87cb3b105c4
Jul 28 19:29:37.857813 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.857790    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:37.857957 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:37.857930    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cx4jl" podUID="8965338e-4b6f-4d05-905f-15d353c0ea7f"
Jul 28 19:29:37.868092 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.868048    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5hfx5" event={"ID":"fdf14d1f-4a97-49ed-a0f4-f28fd90e5482","Type":"ContainerStarted","Data":"e142b2eeec9e5933854e5c67ac11e6e9af6803622d0aebe36d994a7b6348e6de"}
Jul 28 19:29:37.869469 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.869442    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tgq2d" event={"ID":"be4dd071-8c56-455a-bb1e-440b5b292f3b","Type":"ContainerStarted","Data":"0db478b894f2a6f6030691a7b7994a2433ec607aebc487801ecbfce23939e891"}
Jul 28 19:29:37.870844 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.870812    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cnk8n" event={"ID":"a00376ce-82e8-47b8-917b-9ed30dd65272","Type":"ContainerStarted","Data":"24dcca221fd43979f268555b9f1077f77b770304160c5d271fa6a790961181a2"}
Jul 28 19:29:37.875723 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.875690    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bsh96" event={"ID":"2acc75c7-ac70-4ca4-b588-dde7608024c3","Type":"ContainerStarted","Data":"fa46e7001fb26cd4cf7d4ba23e1358db5aeced2f243036b9cac6727826970cb5"}
Jul 28 19:29:37.877221 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.877022    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d8492" event={"ID":"babffeed-a4c6-47f8-b0a6-eee191647b19","Type":"ContainerStarted","Data":"cfe6c8d2b6d12db2090c6ccebd6def508fe1cc28b5692d0eca2fb87cb3b105c4"}
Jul 28 19:29:37.878515 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.878461    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-td8fx" event={"ID":"4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5","Type":"ContainerStarted","Data":"09eecef500d0a24f9bda732fc42563e35b0ccabcceb009970474cab8ed91aacb"}
Jul 28 19:29:37.880583 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.880557    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz" event={"ID":"78af26d1-d3f7-4ef9-b8d5-803606097739","Type":"ContainerStarted","Data":"dffdd5e3ad6e85d8b20e25e04d365e1814cd388ed00ded2310ad931b21460def"}
Jul 28 19:29:37.882221 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.882193    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kqtsk" event={"ID":"def32d52-4c44-402a-a3e3-354a1ce6ce1d","Type":"ContainerStarted","Data":"1cec1c7e151a77569c9f68d779b9b74f135c71162940f46dd085a85bbc888197"}
Jul 28 19:29:37.883735 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:37.883712    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf" event={"ID":"112bf4e8-7d1b-4aa1-aba7-6970554df9ab","Type":"ContainerStarted","Data":"2cc608c1b58d6e4f421430bc611ec91bd6776fd00bc2c7216ec047590fc0fa0f"}
Jul 28 19:29:38.429432 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:38.429119    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret\") pod \"global-pull-secret-syncer-vlfs6\" (UID: \"2c007a53-c536-4083-ab29-644788d631ba\") " pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:38.429432 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:38.429420    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs\") pod \"network-metrics-daemon-8b8d6\" (UID: \"303f736b-24a3-4e9b-8904-e4fde37ab7e0\") " pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:38.429656 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:38.429580    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:29:38.429656 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:38.429651    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs podName:303f736b-24a3-4e9b-8904-e4fde37ab7e0 nodeName:}" failed. No retries permitted until 2026-07-28 19:29:40.429629171 +0000 UTC m=+6.017292101 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs") pod "network-metrics-daemon-8b8d6" (UID: "303f736b-24a3-4e9b-8904-e4fde37ab7e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:29:38.429779 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:38.429720    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:29:38.429779 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:38.429754    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret podName:2c007a53-c536-4083-ab29-644788d631ba nodeName:}" failed. No retries permitted until 2026-07-28 19:29:40.429742285 +0000 UTC m=+6.017405219 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret") pod "global-pull-secret-syncer-vlfs6" (UID: "2c007a53-c536-4083-ab29-644788d631ba") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:29:38.530817 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:38.530775    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flv9j\" (UniqueName: \"kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j\") pod \"network-check-target-cx4jl\" (UID: \"8965338e-4b6f-4d05-905f-15d353c0ea7f\") " pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:38.531565 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:38.530983    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:29:38.531565 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:38.531012    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:29:38.531565 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:38.531026    2572 projected.go:196] Error preparing data for projected volume kube-api-access-flv9j for pod openshift-network-diagnostics/network-check-target-cx4jl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:29:38.531565 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:38.531097    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j podName:8965338e-4b6f-4d05-905f-15d353c0ea7f nodeName:}" failed. No retries permitted until 2026-07-28 19:29:40.531078639 +0000 UTC m=+6.118741567 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-flv9j" (UniqueName: "kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j") pod "network-check-target-cx4jl" (UID: "8965338e-4b6f-4d05-905f-15d353c0ea7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:29:38.742700 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:38.742632    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 19:24:36 +0000 UTC" deadline="2028-02-19 08:18:53.96478077 +0000 UTC"
Jul 28 19:29:38.742700 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:38.742694    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13692h49m15.222091221s"
Jul 28 19:29:38.866430 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:38.865667    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:38.866430 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:38.865797    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlfs6" podUID="2c007a53-c536-4083-ab29-644788d631ba"
Jul 28 19:29:38.866430 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:38.866240    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:38.866430 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:38.866353    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:29:38.904611 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:38.904577    2572 generic.go:356] "Generic (PLEG): container finished" podID="7530272d554f2ab529547b87329e5402" containerID="aaf1e10bc2b68aaa45645a31cf447bf29aa1d317a41402f389c5924f94b7d596" exitCode=0
Jul 28 19:29:38.904750 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:38.904667    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal" event={"ID":"7530272d554f2ab529547b87329e5402","Type":"ContainerDied","Data":"aaf1e10bc2b68aaa45645a31cf447bf29aa1d317a41402f389c5924f94b7d596"}
Jul 28 19:29:38.919687 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:38.919255    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal" event={"ID":"70c009bc8352f8c83009ce9d3b8a04d2","Type":"ContainerStarted","Data":"ed04a2743ab2fad822e107f061903c73d6f0bdd0627fa44168a9458fdc775ad0"}
Jul 28 19:29:38.936936 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:38.936888    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal" podStartSLOduration=2.936871648 podStartE2EDuration="2.936871648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:29:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:29:38.936664071 +0000 UTC m=+4.524327018" watchObservedRunningTime="2026-07-28 19:29:38.936871648 +0000 UTC m=+4.524534597"
Jul 28 19:29:39.858744 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:39.858715    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:39.859323 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:39.858830    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cx4jl" podUID="8965338e-4b6f-4d05-905f-15d353c0ea7f"
Jul 28 19:29:39.937018 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:39.936966    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal" event={"ID":"7530272d554f2ab529547b87329e5402","Type":"ContainerStarted","Data":"105cb54d87fcc5ea99a1c6bbaed8f50c90f652e931b68a7bcb6931e247f45e31"}
Jul 28 19:29:40.451504 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:40.451467    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret\") pod \"global-pull-secret-syncer-vlfs6\" (UID: \"2c007a53-c536-4083-ab29-644788d631ba\") " pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:40.451683 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:40.451532    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs\") pod \"network-metrics-daemon-8b8d6\" (UID: \"303f736b-24a3-4e9b-8904-e4fde37ab7e0\") " pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:40.451752 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:40.451723    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:29:40.451808 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:40.451789    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs podName:303f736b-24a3-4e9b-8904-e4fde37ab7e0 nodeName:}" failed. No retries permitted until 2026-07-28 19:29:44.451770333 +0000 UTC m=+10.039433258 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs") pod "network-metrics-daemon-8b8d6" (UID: "303f736b-24a3-4e9b-8904-e4fde37ab7e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:29:40.452296 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:40.452204    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:29:40.452296 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:40.452256    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret podName:2c007a53-c536-4083-ab29-644788d631ba nodeName:}" failed. No retries permitted until 2026-07-28 19:29:44.452240412 +0000 UTC m=+10.039903341 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret") pod "global-pull-secret-syncer-vlfs6" (UID: "2c007a53-c536-4083-ab29-644788d631ba") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:29:40.552594 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:40.552434    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flv9j\" (UniqueName: \"kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j\") pod \"network-check-target-cx4jl\" (UID: \"8965338e-4b6f-4d05-905f-15d353c0ea7f\") " pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:40.552795 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:40.552698    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:29:40.552795 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:40.552726    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:29:40.552795 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:40.552740    2572 projected.go:196] Error preparing data for projected volume kube-api-access-flv9j for pod openshift-network-diagnostics/network-check-target-cx4jl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:29:40.552795 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:40.552795    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j podName:8965338e-4b6f-4d05-905f-15d353c0ea7f nodeName:}" failed. No retries permitted until 2026-07-28 19:29:44.552777194 +0000 UTC m=+10.140440133 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-flv9j" (UniqueName: "kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j") pod "network-check-target-cx4jl" (UID: "8965338e-4b6f-4d05-905f-15d353c0ea7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:29:40.858876 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:40.858196    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:40.858876 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:40.858229    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:40.858876 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:40.858363    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:29:40.858876 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:40.858794    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlfs6" podUID="2c007a53-c536-4083-ab29-644788d631ba"
Jul 28 19:29:41.858566 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:41.858531    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:41.858738 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:41.858668    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cx4jl" podUID="8965338e-4b6f-4d05-905f-15d353c0ea7f"
Jul 28 19:29:42.858528 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:42.858245    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:42.858528 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:42.858306    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:42.858528 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:42.858449    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlfs6" podUID="2c007a53-c536-4083-ab29-644788d631ba"
Jul 28 19:29:42.859087 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:42.858552    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:29:43.858757 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:43.858704    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:43.859236 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:43.858832    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cx4jl" podUID="8965338e-4b6f-4d05-905f-15d353c0ea7f"
Jul 28 19:29:44.484697 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:44.484660    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret\") pod \"global-pull-secret-syncer-vlfs6\" (UID: \"2c007a53-c536-4083-ab29-644788d631ba\") " pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:44.484887 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:44.484715    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs\") pod \"network-metrics-daemon-8b8d6\" (UID: \"303f736b-24a3-4e9b-8904-e4fde37ab7e0\") " pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:44.484887 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:44.484861    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:29:44.485012 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:44.484918    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs podName:303f736b-24a3-4e9b-8904-e4fde37ab7e0 nodeName:}" failed. No retries permitted until 2026-07-28 19:29:52.48490069 +0000 UTC m=+18.072563626 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs") pod "network-metrics-daemon-8b8d6" (UID: "303f736b-24a3-4e9b-8904-e4fde37ab7e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:29:44.485333 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:44.485311    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:29:44.485434 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:44.485368    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret podName:2c007a53-c536-4083-ab29-644788d631ba nodeName:}" failed. No retries permitted until 2026-07-28 19:29:52.48535318 +0000 UTC m=+18.073016110 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret") pod "global-pull-secret-syncer-vlfs6" (UID: "2c007a53-c536-4083-ab29-644788d631ba") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:29:44.585641 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:44.585300    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flv9j\" (UniqueName: \"kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j\") pod \"network-check-target-cx4jl\" (UID: \"8965338e-4b6f-4d05-905f-15d353c0ea7f\") " pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:44.585641 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:44.585484    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:29:44.585641 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:44.585513    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:29:44.585641 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:44.585526    2572 projected.go:196] Error preparing data for projected volume kube-api-access-flv9j for pod openshift-network-diagnostics/network-check-target-cx4jl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:29:44.585641 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:44.585586    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j podName:8965338e-4b6f-4d05-905f-15d353c0ea7f nodeName:}" failed. No retries permitted until 2026-07-28 19:29:52.58556653 +0000 UTC m=+18.173229456 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-flv9j" (UniqueName: "kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j") pod "network-check-target-cx4jl" (UID: "8965338e-4b6f-4d05-905f-15d353c0ea7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:29:44.858609 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:44.858528    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:44.859661 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:44.859604    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlfs6" podUID="2c007a53-c536-4083-ab29-644788d631ba"
Jul 28 19:29:44.860034 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:44.859966    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:44.860106 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:44.860088    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:29:45.859010 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:45.858791    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:45.859010 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:45.858920    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cx4jl" podUID="8965338e-4b6f-4d05-905f-15d353c0ea7f"
Jul 28 19:29:46.858145 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:46.858104    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:46.858145 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:46.858120    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:46.858632 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:46.858234    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlfs6" podUID="2c007a53-c536-4083-ab29-644788d631ba"
Jul 28 19:29:46.858632 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:46.858367    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:29:47.857883 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:47.857854    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:47.858061 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:47.857952    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cx4jl" podUID="8965338e-4b6f-4d05-905f-15d353c0ea7f"
Jul 28 19:29:48.858692 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:48.858658    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:48.859162 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:48.858791    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlfs6" podUID="2c007a53-c536-4083-ab29-644788d631ba"
Jul 28 19:29:48.859162 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:48.858851    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:48.859162 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:48.858988    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:29:49.858801 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:49.858772    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:49.859293 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:49.858919    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cx4jl" podUID="8965338e-4b6f-4d05-905f-15d353c0ea7f"
Jul 28 19:29:50.857938 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:50.857819    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:50.857938 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:50.857848    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:50.858200 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:50.857942    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlfs6" podUID="2c007a53-c536-4083-ab29-644788d631ba"
Jul 28 19:29:50.858200 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:50.858036    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:29:51.858392 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:51.858369    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:51.858730 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:51.858467    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cx4jl" podUID="8965338e-4b6f-4d05-905f-15d353c0ea7f"
Jul 28 19:29:52.547404 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:52.547374    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret\") pod \"global-pull-secret-syncer-vlfs6\" (UID: \"2c007a53-c536-4083-ab29-644788d631ba\") " pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:52.547607 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:52.547413    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs\") pod \"network-metrics-daemon-8b8d6\" (UID: \"303f736b-24a3-4e9b-8904-e4fde37ab7e0\") " pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:52.547607 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:52.547538    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:29:52.547702 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:52.547608    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret podName:2c007a53-c536-4083-ab29-644788d631ba nodeName:}" failed. No retries permitted until 2026-07-28 19:30:08.547586005 +0000 UTC m=+34.135248944 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret") pod "global-pull-secret-syncer-vlfs6" (UID: "2c007a53-c536-4083-ab29-644788d631ba") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:29:52.547702 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:52.547546    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:29:52.547702 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:52.547667    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs podName:303f736b-24a3-4e9b-8904-e4fde37ab7e0 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:08.547648698 +0000 UTC m=+34.135311623 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs") pod "network-metrics-daemon-8b8d6" (UID: "303f736b-24a3-4e9b-8904-e4fde37ab7e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:29:52.648448 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:52.648411    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flv9j\" (UniqueName: \"kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j\") pod \"network-check-target-cx4jl\" (UID: \"8965338e-4b6f-4d05-905f-15d353c0ea7f\") " pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:52.648598 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:52.648557    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:29:52.648598 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:52.648581    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:29:52.648598 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:52.648594    2572 projected.go:196] Error preparing data for projected volume kube-api-access-flv9j for pod openshift-network-diagnostics/network-check-target-cx4jl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:29:52.648718 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:52.648644    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j podName:8965338e-4b6f-4d05-905f-15d353c0ea7f nodeName:}" failed. No retries permitted until 2026-07-28 19:30:08.648628223 +0000 UTC m=+34.236291149 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-flv9j" (UniqueName: "kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j") pod "network-check-target-cx4jl" (UID: "8965338e-4b6f-4d05-905f-15d353c0ea7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:29:52.858954 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:52.858870    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:52.859441 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:52.859014    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlfs6" podUID="2c007a53-c536-4083-ab29-644788d631ba"
Jul 28 19:29:52.859441 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:52.859060    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:52.859441 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:52.859165    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:29:53.858514 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:53.858477    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:53.858691 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:53.858603    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cx4jl" podUID="8965338e-4b6f-4d05-905f-15d353c0ea7f"
Jul 28 19:29:54.859316 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:54.859282    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:54.859769 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:54.859393    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlfs6" podUID="2c007a53-c536-4083-ab29-644788d631ba"
Jul 28 19:29:54.859769 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:54.859438    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:54.859769 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:54.859526    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:29:55.857939 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:55.857881    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:55.858041 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:55.858004    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cx4jl" podUID="8965338e-4b6f-4d05-905f-15d353c0ea7f"
Jul 28 19:29:55.967664 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:55.967505    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5hfx5" event={"ID":"fdf14d1f-4a97-49ed-a0f4-f28fd90e5482","Type":"ContainerStarted","Data":"b35c0489b7603d666b64288850abff002e2ce85827471c3adf56ba7360938434"}
Jul 28 19:29:55.968981 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:55.968959    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tgq2d" event={"ID":"be4dd071-8c56-455a-bb1e-440b5b292f3b","Type":"ContainerStarted","Data":"44855707ec22c6c4e82176c2647abe3b82382aee449e1dcf24d9cad834007ccc"}
Jul 28 19:29:55.970308 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:55.970281    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bsh96" event={"ID":"2acc75c7-ac70-4ca4-b588-dde7608024c3","Type":"ContainerStarted","Data":"1e87b9e489412e1b46c2ddd3394c74274967c513e1174395b29b0c68f3339829"}
Jul 28 19:29:55.971620 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:55.971601    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf" event={"ID":"112bf4e8-7d1b-4aa1-aba7-6970554df9ab","Type":"ContainerStarted","Data":"b0cb30b53f8902012a60fe316cc4860b298f459665c23613c8c24a4edf8ae45d"}
Jul 28 19:29:55.982165 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:55.982128    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5hfx5" podStartSLOduration=6.294144892 podStartE2EDuration="20.982117901s" totalImagesPullingTime="14.687973009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:29:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:29:55.982113023 +0000 UTC m=+21.569775969" watchObservedRunningTime="2026-07-28 19:29:55.982117901 +0000 UTC m=+21.569780847"
Jul 28 19:29:55.982368 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:55.982350    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal" podStartSLOduration=19.982343497 podStartE2EDuration="19.982343497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:29:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:29:39.952371614 +0000 UTC m=+5.540034562" watchObservedRunningTime="2026-07-28 19:29:55.982343497 +0000 UTC m=+21.570006442"
Jul 28 19:29:56.022769 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:56.022656    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tgq2d" podStartSLOduration=2.37834813 podStartE2EDuration="21.022640732s" totalImagesPullingTime="18.644292602s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:29:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:29:56.022530694 +0000 UTC m=+21.610193640" watchObservedRunningTime="2026-07-28 19:29:56.022640732 +0000 UTC m=+21.610303679"
Jul 28 19:29:56.858206 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:56.858096    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:56.858291 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:56.858161    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:56.858291 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:56.858275    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlfs6" podUID="2c007a53-c536-4083-ab29-644788d631ba"
Jul 28 19:29:56.858422 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:56.858400    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:29:56.974624 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:56.974596    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cnk8n" event={"ID":"a00376ce-82e8-47b8-917b-9ed30dd65272","Type":"ContainerStarted","Data":"64208e7ab95f8a667c5378c251f91a8a5f69b18f3d716b845e296b025cc6d0ca"}
Jul 28 19:29:56.976031 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:56.976009    2572 generic.go:356] "Generic (PLEG): container finished" podID="2acc75c7-ac70-4ca4-b588-dde7608024c3" containerID="1e87b9e489412e1b46c2ddd3394c74274967c513e1174395b29b0c68f3339829" exitCode=0
Jul 28 19:29:56.976136 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:56.976108    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bsh96" event={"ID":"2acc75c7-ac70-4ca4-b588-dde7608024c3","Type":"ContainerDied","Data":"1e87b9e489412e1b46c2ddd3394c74274967c513e1174395b29b0c68f3339829"}
Jul 28 19:29:56.977545 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:56.977520    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-td8fx" event={"ID":"4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5","Type":"ContainerStarted","Data":"9da7036383ea61b0bc0752ff831daf388440bb559777350ea5e30f20fca03bb0"}
Jul 28 19:29:56.979438 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:56.979414    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz" event={"ID":"78af26d1-d3f7-4ef9-b8d5-803606097739","Type":"ContainerStarted","Data":"b0d45b02095ce846ed9a538e019555557fab771f90010f75741c2f93d5dcefca"}
Jul 28 19:29:56.979524 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:56.979485    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz" event={"ID":"78af26d1-d3f7-4ef9-b8d5-803606097739","Type":"ContainerStarted","Data":"6f55888fc8cf24ff1dda5d5281d7c419677ee09ee22d40d88b8d5ec85552fb4f"}
Jul 28 19:29:56.980932 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:56.980908    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kqtsk" event={"ID":"def32d52-4c44-402a-a3e3-354a1ce6ce1d","Type":"ContainerStarted","Data":"ffbd85259c1045c5e987180106f290b0b4829059a86f8253754b5fe7f4b73d1e"}
Jul 28 19:29:56.991116 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:56.991043    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cnk8n" podStartSLOduration=3.288226582 podStartE2EDuration="21.991025816s" totalImagesPullingTime="18.702799234s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:29:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:29:56.990564621 +0000 UTC m=+22.578227572" watchObservedRunningTime="2026-07-28 19:29:56.991025816 +0000 UTC m=+22.578688763"
Jul 28 19:29:57.024833 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:57.024811    2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 19:29:57.025316 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:57.025282    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kqtsk" podStartSLOduration=3.362371641 podStartE2EDuration="22.025271492s" totalImagesPullingTime="18.662899851s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:29:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:29:57.025047949 +0000 UTC m=+22.612710899" watchObservedRunningTime="2026-07-28 19:29:57.025271492 +0000 UTC m=+22.612934462"
Jul 28 19:29:57.038481 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:57.038447    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-td8fx" podStartSLOduration=4.068473119 podStartE2EDuration="22.038435238s" totalImagesPullingTime="17.969962119s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:29:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:29:57.037954503 +0000 UTC m=+22.625617449" watchObservedRunningTime="2026-07-28 19:29:57.038435238 +0000 UTC m=+22.626098184"
Jul 28 19:29:57.370766 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:57.370708    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5hfx5"
Jul 28 19:29:57.371728 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:57.371710    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5hfx5"
Jul 28 19:29:57.770579 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:57.770416    2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T19:29:57.024830818Z","UUID":"1832b7ee-bc90-4140-970e-4c35294f01a3","Handler":null,"Name":"","Endpoint":""}
Jul 28 19:29:57.772084 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:57.772037    2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 19:29:57.772196 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:57.772096    2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 19:29:57.857822 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:57.857791    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:57.857969 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:57.857927    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cx4jl" podUID="8965338e-4b6f-4d05-905f-15d353c0ea7f"
Jul 28 19:29:57.984995 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:57.984961    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d8492" event={"ID":"babffeed-a4c6-47f8-b0a6-eee191647b19","Type":"ContainerStarted","Data":"6b431ae283c2b2cd60c0116d08cafc30aaf33b0af46b0d38ced0d19c6c782b2c"}
Jul 28 19:29:57.988410 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:57.988382    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz" event={"ID":"78af26d1-d3f7-4ef9-b8d5-803606097739","Type":"ContainerStarted","Data":"de89d778fb707216db57ade5d8ca6c4c19a3db3df47111ac6d9337b0f29ed3f2"}
Jul 28 19:29:57.988539 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:57.988461    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz" event={"ID":"78af26d1-d3f7-4ef9-b8d5-803606097739","Type":"ContainerStarted","Data":"3ae6ffad8cc073919be0760e03d87d2e0e276f0bc40cd7c2d39fa5cbb697a928"}
Jul 28 19:29:57.988539 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:57.988511    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz" event={"ID":"78af26d1-d3f7-4ef9-b8d5-803606097739","Type":"ContainerStarted","Data":"cef6482c500d78c44d3dab87c18bcb903b19e45870396a5f0665b6259ac455b0"}
Jul 28 19:29:57.988636 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:57.988560    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz" event={"ID":"78af26d1-d3f7-4ef9-b8d5-803606097739","Type":"ContainerStarted","Data":"7c079f2386c7a49560236f0cafbd8e8bf5a566543dd0f09fa86c2b7a57425e7d"}
Jul 28 19:29:57.990502 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:57.990474    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf" event={"ID":"112bf4e8-7d1b-4aa1-aba7-6970554df9ab","Type":"ContainerStarted","Data":"f1077eaf2cb444e1f447a9e2a4adec48d32db3d16fb7502983c4a196bbda702d"}
Jul 28 19:29:57.999181 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:57.999143    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-d8492" podStartSLOduration=5.038770996 podStartE2EDuration="22.999131651s" totalImagesPullingTime="17.960360655s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:29:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:29:57.998783832 +0000 UTC m=+23.586446779" watchObservedRunningTime="2026-07-28 19:29:57.999131651 +0000 UTC m=+23.586794596"
Jul 28 19:29:58.857907 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:58.857879    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:29:58.858209 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:58.857998    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlfs6" podUID="2c007a53-c536-4083-ab29-644788d631ba"
Jul 28 19:29:58.858209 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:58.858091    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:29:58.858329 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:58.858237    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:29:58.995719 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:58.995685    2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 19:29:58.996343 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:58.996313    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf" event={"ID":"112bf4e8-7d1b-4aa1-aba7-6970554df9ab","Type":"ContainerStarted","Data":"ada35b13d80b29a140ac362ef56ee4a108998555c71df152fc3d99aeda12f884"}
Jul 28 19:29:59.013458 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:59.013411    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzcrf" podStartSLOduration=3.418153468 podStartE2EDuration="24.013393776s" totalImagesPullingTime="20.595240308s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:29:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:29:59.013326108 +0000 UTC m=+24.600989054" watchObservedRunningTime="2026-07-28 19:29:59.013393776 +0000 UTC m=+24.601056721"
Jul 28 19:29:59.858361 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:29:59.858111    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:29:59.858521 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:29:59.858418    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cx4jl" podUID="8965338e-4b6f-4d05-905f-15d353c0ea7f"
Jul 28 19:30:00.858537 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:00.858502    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:30:00.858537 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:00.858529    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:30:00.859057 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:00.858674    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlfs6" podUID="2c007a53-c536-4083-ab29-644788d631ba"
Jul 28 19:30:00.859057 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:00.858790    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:30:01.858670 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:01.858642    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:30:01.859212 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:01.858731    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cx4jl" podUID="8965338e-4b6f-4d05-905f-15d353c0ea7f"
Jul 28 19:30:02.005189 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:02.005157    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz" event={"ID":"78af26d1-d3f7-4ef9-b8d5-803606097739","Type":"ContainerStarted","Data":"4ba3822eeb24dc698e848125866dac1386937300bd2f9eb3e635ca49b405ca12"}
Jul 28 19:30:02.006896 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:02.006874    2572 generic.go:356] "Generic (PLEG): container finished" podID="2acc75c7-ac70-4ca4-b588-dde7608024c3" containerID="955fe95361ee0969d925f64d9d9dd2fd7d0eb751ae3400c43acbaed0674b6bb3" exitCode=0
Jul 28 19:30:02.007013 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:02.006906    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bsh96" event={"ID":"2acc75c7-ac70-4ca4-b588-dde7608024c3","Type":"ContainerDied","Data":"955fe95361ee0969d925f64d9d9dd2fd7d0eb751ae3400c43acbaed0674b6bb3"}
Jul 28 19:30:02.858004 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:02.857976    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:30:02.858153 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:02.858105    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:30:02.858202 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:02.858157    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:30:02.858271 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:02.858254    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlfs6" podUID="2c007a53-c536-4083-ab29-644788d631ba"
Jul 28 19:30:03.858591 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:03.858429    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:30:03.859056 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:03.858656    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cx4jl" podUID="8965338e-4b6f-4d05-905f-15d353c0ea7f"
Jul 28 19:30:04.014857 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:04.014821    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz" event={"ID":"78af26d1-d3f7-4ef9-b8d5-803606097739","Type":"ContainerStarted","Data":"719d4bb94be10aa66a8335bf5faeaad14c70eb3762e6a5d9c56569fd58acee1a"}
Jul 28 19:30:04.015062 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:04.015043    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:30:04.017114 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:04.017091    2572 generic.go:356] "Generic (PLEG): container finished" podID="2acc75c7-ac70-4ca4-b588-dde7608024c3" containerID="07647752ad57c153923578ae38a39602bdbdaab892d08d2d807dd9882488c9b5" exitCode=0
Jul 28 19:30:04.017114 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:04.017121    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bsh96" event={"ID":"2acc75c7-ac70-4ca4-b588-dde7608024c3","Type":"ContainerDied","Data":"07647752ad57c153923578ae38a39602bdbdaab892d08d2d807dd9882488c9b5"}
Jul 28 19:30:04.029969 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:04.029950    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:30:04.042355 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:04.042314    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz" podStartSLOduration=9.668135209999999 podStartE2EDuration="29.042302668s" totalImagesPullingTime="19.374167458s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:29:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:30:04.042019478 +0000 UTC m=+29.629682428" watchObservedRunningTime="2026-07-28 19:30:04.042302668 +0000 UTC m=+29.629965614"
Jul 28 19:30:04.859574 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:04.859543    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:30:04.859956 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:04.859653    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlfs6" podUID="2c007a53-c536-4083-ab29-644788d631ba"
Jul 28 19:30:04.859956 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:04.859667    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:30:04.859956 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:04.859803    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:30:05.020626 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:05.020477    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:30:05.020626 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:05.020561    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:30:05.037212 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:05.037183    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:30:05.112527 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:05.112467    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5hfx5"
Jul 28 19:30:05.112640 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:05.112583    2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 19:30:05.113167 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:05.113131    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5hfx5"
Jul 28 19:30:05.273211 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:05.272998    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vlfs6"]
Jul 28 19:30:05.273379 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:05.273291    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:30:05.273444 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:05.273401    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlfs6" podUID="2c007a53-c536-4083-ab29-644788d631ba"
Jul 28 19:30:05.276994 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:05.276968    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8b8d6"]
Jul 28 19:30:05.277668 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:05.277058    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:30:05.277668 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:05.277179    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:30:05.277784 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:05.277716    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cx4jl"]
Jul 28 19:30:05.277840 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:05.277795    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:30:05.277899 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:05.277864    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cx4jl" podUID="8965338e-4b6f-4d05-905f-15d353c0ea7f"
Jul 28 19:30:06.024493 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:06.024463    2572 generic.go:356] "Generic (PLEG): container finished" podID="2acc75c7-ac70-4ca4-b588-dde7608024c3" containerID="48a3a42866a3c1eb0d9910e3fe4708a3795c6f70880bea755ff13cbe83261d37" exitCode=0
Jul 28 19:30:06.024988 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:06.024543    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bsh96" event={"ID":"2acc75c7-ac70-4ca4-b588-dde7608024c3","Type":"ContainerDied","Data":"48a3a42866a3c1eb0d9910e3fe4708a3795c6f70880bea755ff13cbe83261d37"}
Jul 28 19:30:06.858209 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:06.858177    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:30:06.858357 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:06.858179    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:30:06.858357 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:06.858287    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlfs6" podUID="2c007a53-c536-4083-ab29-644788d631ba"
Jul 28 19:30:06.858431 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:06.858382    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:30:06.858431 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:06.858182    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:30:06.858515 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:06.858464    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cx4jl" podUID="8965338e-4b6f-4d05-905f-15d353c0ea7f"
Jul 28 19:30:08.569672 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:08.569626    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs\") pod \"network-metrics-daemon-8b8d6\" (UID: \"303f736b-24a3-4e9b-8904-e4fde37ab7e0\") " pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:30:08.570218 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:08.569723    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret\") pod \"global-pull-secret-syncer-vlfs6\" (UID: \"2c007a53-c536-4083-ab29-644788d631ba\") " pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:30:08.570218 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:08.569786    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:30:08.570218 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:08.569834    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:30:08.570218 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:08.569866    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs podName:303f736b-24a3-4e9b-8904-e4fde37ab7e0 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:40.569845268 +0000 UTC m=+66.157508206 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs") pod "network-metrics-daemon-8b8d6" (UID: "303f736b-24a3-4e9b-8904-e4fde37ab7e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:30:08.570218 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:08.569885    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret podName:2c007a53-c536-4083-ab29-644788d631ba nodeName:}" failed. No retries permitted until 2026-07-28 19:30:40.569875895 +0000 UTC m=+66.157538819 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret") pod "global-pull-secret-syncer-vlfs6" (UID: "2c007a53-c536-4083-ab29-644788d631ba") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:30:08.670007 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:08.669979    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flv9j\" (UniqueName: \"kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j\") pod \"network-check-target-cx4jl\" (UID: \"8965338e-4b6f-4d05-905f-15d353c0ea7f\") " pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:30:08.670176 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:08.670157    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:30:08.670228 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:08.670184    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:30:08.670228 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:08.670197    2572 projected.go:196] Error preparing data for projected volume kube-api-access-flv9j for pod openshift-network-diagnostics/network-check-target-cx4jl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:30:08.670289 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:08.670245    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j podName:8965338e-4b6f-4d05-905f-15d353c0ea7f nodeName:}" failed. No retries permitted until 2026-07-28 19:30:40.670231706 +0000 UTC m=+66.257894630 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-flv9j" (UniqueName: "kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j") pod "network-check-target-cx4jl" (UID: "8965338e-4b6f-4d05-905f-15d353c0ea7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:30:08.858294 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:08.858222    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:30:08.858294 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:08.858237    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:30:08.858489 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:08.858222    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:30:08.858489 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:08.858360    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlfs6" podUID="2c007a53-c536-4083-ab29-644788d631ba"
Jul 28 19:30:08.858489 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:08.858445    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cx4jl" podUID="8965338e-4b6f-4d05-905f-15d353c0ea7f"
Jul 28 19:30:08.858635 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:08.858548    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:30:09.728684 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.728604    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeReady"
Jul 28 19:30:09.729165 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.728753    2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 19:30:09.767081 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.767033    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"]
Jul 28 19:30:09.770735 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.770713    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-9fc54f48b-9jtb2"]
Jul 28 19:30:09.770852 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.770837    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:30:09.774893 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.773474    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 28 19:30:09.774893 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.773553    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-phlnm\""
Jul 28 19:30:09.774893 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.773898    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 28 19:30:09.776318 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.776296    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.778874 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.778852    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5qrcf\""
Jul 28 19:30:09.779633 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.779584    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 19:30:09.779747 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.779723    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 19:30:09.779843 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.779827    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"]
Jul 28 19:30:09.780175 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.780143    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 19:30:09.785658 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.785639    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 19:30:09.787415 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.787363    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6wqlb"]
Jul 28 19:30:09.790499 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.790479    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tpngt"]
Jul 28 19:30:09.790617 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.790597    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:30:09.793141 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.793121    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 19:30:09.793238 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.793123    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 19:30:09.793238 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.793164    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pwrtc\""
Jul 28 19:30:09.793588 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.793570    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tpngt"
Jul 28 19:30:09.796387 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.796260    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 19:30:09.796806 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.796595    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-75bnq\""
Jul 28 19:30:09.796806 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.796630    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 19:30:09.796806 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.796650    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 19:30:09.805956 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.804728    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9fc54f48b-9jtb2"]
Jul 28 19:30:09.806218 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.806196    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tpngt"]
Jul 28 19:30:09.806315 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.806236    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6wqlb"]
Jul 28 19:30:09.833766 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.833741    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:09.879179 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.879155    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.879282 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.879193    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f427e901-fd2a-4d35-a853-fe83fabbca35-image-registry-private-configuration\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.879282 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.879225    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-certificates\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.879377 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.879298    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f427e901-fd2a-4d35-a853-fe83fabbca35-ca-trust-extracted\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.879377 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.879330    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f427e901-fd2a-4d35-a853-fe83fabbca35-trusted-ca\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.879377 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.879352    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f427e901-fd2a-4d35-a853-fe83fabbca35-installation-pull-secrets\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.879505 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.879376    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5x7t\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-kube-api-access-x5x7t\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.879505 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.879477    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-txhmp\" (UID: \"0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:30:09.879505 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.879497    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-bound-sa-token\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.879626 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.879526    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-txhmp\" (UID: \"0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:30:09.944043 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.944016    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:09.980717 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.980655    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5x7t\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-kube-api-access-x5x7t\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.980833 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.980725    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:30:09.980833 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.980753    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-txhmp\" (UID: \"0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:30:09.980833 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.980778    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-bound-sa-token\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.980833 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.980817    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-txhmp\" (UID: \"0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:30:09.981035 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.980839    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5c7f30b-8bac-4127-9e0a-44c45b92e47b-cert\") pod \"ingress-canary-tpngt\" (UID: \"a5c7f30b-8bac-4127-9e0a-44c45b92e47b\") " pod="openshift-ingress-canary/ingress-canary-tpngt"
Jul 28 19:30:09.981035 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.980866    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.981035 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.980891    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45qtv\" (UniqueName: \"kubernetes.io/projected/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-kube-api-access-45qtv\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:30:09.981035 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.980913    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-config-volume\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:30:09.981035 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.980941    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f427e901-fd2a-4d35-a853-fe83fabbca35-image-registry-private-configuration\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.981035 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.981005    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltfjf\" (UniqueName: \"kubernetes.io/projected/a5c7f30b-8bac-4127-9e0a-44c45b92e47b-kube-api-access-ltfjf\") pod \"ingress-canary-tpngt\" (UID: \"a5c7f30b-8bac-4127-9e0a-44c45b92e47b\") " pod="openshift-ingress-canary/ingress-canary-tpngt"
Jul 28 19:30:09.981351 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:09.981010    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:30:09.981351 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:09.981056    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9fc54f48b-9jtb2: secret "image-registry-tls" not found
Jul 28 19:30:09.981351 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:09.981135    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls podName:f427e901-fd2a-4d35-a853-fe83fabbca35 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:10.481115831 +0000 UTC m=+36.068778766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls") pod "image-registry-9fc54f48b-9jtb2" (UID: "f427e901-fd2a-4d35-a853-fe83fabbca35") : secret "image-registry-tls" not found
Jul 28 19:30:09.981351 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:09.981035    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:30:09.981351 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:09.981247    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert podName:0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:10.481234313 +0000 UTC m=+36.068897237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-txhmp" (UID: "0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5") : secret "networking-console-plugin-cert" not found
Jul 28 19:30:09.981351 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.981269    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-certificates\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.981655 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.981350    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-tmp-dir\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:30:09.981655 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.981403    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f427e901-fd2a-4d35-a853-fe83fabbca35-ca-trust-extracted\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.981655 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.981424    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f427e901-fd2a-4d35-a853-fe83fabbca35-trusted-ca\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.981655 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.981456    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f427e901-fd2a-4d35-a853-fe83fabbca35-installation-pull-secrets\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.981655 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.981582    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-txhmp\" (UID: \"0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:30:09.981967 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.981944    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f427e901-fd2a-4d35-a853-fe83fabbca35-ca-trust-extracted\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.982046 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.981964    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-certificates\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.982318 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.982296    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f427e901-fd2a-4d35-a853-fe83fabbca35-trusted-ca\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.986108 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.986085    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f427e901-fd2a-4d35-a853-fe83fabbca35-image-registry-private-configuration\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.986258 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.986110    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f427e901-fd2a-4d35-a853-fe83fabbca35-installation-pull-secrets\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.990904 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.990618    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5x7t\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-kube-api-access-x5x7t\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:09.991183 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:09.991157    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-bound-sa-token\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:10.045055 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.045024    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:10.082036 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.082010    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:30:10.082154 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.082062    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5c7f30b-8bac-4127-9e0a-44c45b92e47b-cert\") pod \"ingress-canary-tpngt\" (UID: \"a5c7f30b-8bac-4127-9e0a-44c45b92e47b\") " pod="openshift-ingress-canary/ingress-canary-tpngt"
Jul 28 19:30:10.082154 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.082121    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45qtv\" (UniqueName: \"kubernetes.io/projected/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-kube-api-access-45qtv\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:30:10.082154 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.082149    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-config-volume\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:30:10.082311 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:10.082159    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:30:10.082311 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.082187    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltfjf\" (UniqueName: \"kubernetes.io/projected/a5c7f30b-8bac-4127-9e0a-44c45b92e47b-kube-api-access-ltfjf\") pod \"ingress-canary-tpngt\" (UID: \"a5c7f30b-8bac-4127-9e0a-44c45b92e47b\") " pod="openshift-ingress-canary/ingress-canary-tpngt"
Jul 28 19:30:10.082311 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:10.082231    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls podName:eee9f5fc-fb26-4766-86a8-2b2b8b0056c5 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:10.582211311 +0000 UTC m=+36.169874239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls") pod "dns-default-6wqlb" (UID: "eee9f5fc-fb26-4766-86a8-2b2b8b0056c5") : secret "dns-default-metrics-tls" not found
Jul 28 19:30:10.082311 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.082255    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-tmp-dir\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:30:10.082627 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.082607    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-tmp-dir\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:30:10.082849 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.082827    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-config-volume\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:30:10.084722 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.084701    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5c7f30b-8bac-4127-9e0a-44c45b92e47b-cert\") pod \"ingress-canary-tpngt\" (UID: \"a5c7f30b-8bac-4127-9e0a-44c45b92e47b\") " pod="openshift-ingress-canary/ingress-canary-tpngt"
Jul 28 19:30:10.090319 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.090296    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45qtv\" (UniqueName: \"kubernetes.io/projected/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-kube-api-access-45qtv\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:30:10.090888 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.090866    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltfjf\" (UniqueName: \"kubernetes.io/projected/a5c7f30b-8bac-4127-9e0a-44c45b92e47b-kube-api-access-ltfjf\") pod \"ingress-canary-tpngt\" (UID: \"a5c7f30b-8bac-4127-9e0a-44c45b92e47b\") " pod="openshift-ingress-canary/ingress-canary-tpngt"
Jul 28 19:30:10.109720 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.109695    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tpngt"
Jul 28 19:30:10.302813 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.302668    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tpngt"]
Jul 28 19:30:10.305754 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:30:10.305720    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5c7f30b_8bac_4127_9e0a_44c45b92e47b.slice/crio-f8a8cb1a1ce27569f6dcc34631cff847903aeb04147fa206f58a8f887f28ee80 WatchSource:0}: Error finding container f8a8cb1a1ce27569f6dcc34631cff847903aeb04147fa206f58a8f887f28ee80: Status 404 returned error can't find the container with id f8a8cb1a1ce27569f6dcc34631cff847903aeb04147fa206f58a8f887f28ee80
Jul 28 19:30:10.485353 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.485292    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-txhmp\" (UID: \"0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:30:10.485353 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.485331    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:10.485510 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:10.485443    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:30:10.485510 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:10.485468    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:30:10.485510 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:10.485484    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9fc54f48b-9jtb2: secret "image-registry-tls" not found
Jul 28 19:30:10.485625 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:10.485515    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert podName:0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:11.485498735 +0000 UTC m=+37.073161660 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-txhmp" (UID: "0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5") : secret "networking-console-plugin-cert" not found
Jul 28 19:30:10.485625 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:10.485545    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls podName:f427e901-fd2a-4d35-a853-fe83fabbca35 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:11.485527923 +0000 UTC m=+37.073190866 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls") pod "image-registry-9fc54f48b-9jtb2" (UID: "f427e901-fd2a-4d35-a853-fe83fabbca35") : secret "image-registry-tls" not found
Jul 28 19:30:10.585812 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.585787    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:30:10.585937 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:10.585924    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:30:10.585982 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:10.585972    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls podName:eee9f5fc-fb26-4766-86a8-2b2b8b0056c5 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:11.585957259 +0000 UTC m=+37.173620191 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls") pod "dns-default-6wqlb" (UID: "eee9f5fc-fb26-4766-86a8-2b2b8b0056c5") : secret "dns-default-metrics-tls" not found
Jul 28 19:30:10.807792 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.807764    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:10.858134 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.858106    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:30:10.858134 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.858119    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:30:10.858278 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.858108    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:30:10.860700 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.860674    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 19:30:10.860828 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.860705    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b8krh\""
Jul 28 19:30:10.860828 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.860710    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 19:30:10.861029 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.861014    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 19:30:10.861621 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.861604    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zz5d7\""
Jul 28 19:30:10.861712 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.861635    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 19:30:10.867201 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:10.867180    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:11.037179 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:11.037153    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tpngt" event={"ID":"a5c7f30b-8bac-4127-9e0a-44c45b92e47b","Type":"ContainerStarted","Data":"f8a8cb1a1ce27569f6dcc34631cff847903aeb04147fa206f58a8f887f28ee80"}
Jul 28 19:30:11.039686 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:11.039664    2572 generic.go:356] "Generic (PLEG): container finished" podID="2acc75c7-ac70-4ca4-b588-dde7608024c3" containerID="1fbead79840f4065e7b588128b072f056346feddede3439c175fa7f37ab71659" exitCode=0
Jul 28 19:30:11.039765 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:11.039691    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bsh96" event={"ID":"2acc75c7-ac70-4ca4-b588-dde7608024c3","Type":"ContainerDied","Data":"1fbead79840f4065e7b588128b072f056346feddede3439c175fa7f37ab71659"}
Jul 28 19:30:11.492849 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:11.492789    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-txhmp\" (UID: \"0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:30:11.492849 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:11.492821    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:11.493004 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:11.492926    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:30:11.493004 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:11.492944    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9fc54f48b-9jtb2: secret "image-registry-tls" not found
Jul 28 19:30:11.493004 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:11.492984    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls podName:f427e901-fd2a-4d35-a853-fe83fabbca35 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:13.492970893 +0000 UTC m=+39.080633816 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls") pod "image-registry-9fc54f48b-9jtb2" (UID: "f427e901-fd2a-4d35-a853-fe83fabbca35") : secret "image-registry-tls" not found
Jul 28 19:30:11.493004 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:11.492927    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:30:11.493163 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:11.493048    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert podName:0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:13.493035452 +0000 UTC m=+39.080698375 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-txhmp" (UID: "0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5") : secret "networking-console-plugin-cert" not found
Jul 28 19:30:11.593431 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:11.593405    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:30:11.593558 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:11.593540    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:30:11.593600 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:11.593595    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls podName:eee9f5fc-fb26-4766-86a8-2b2b8b0056c5 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:13.593581992 +0000 UTC m=+39.181244920 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls") pod "dns-default-6wqlb" (UID: "eee9f5fc-fb26-4766-86a8-2b2b8b0056c5") : secret "dns-default-metrics-tls" not found
Jul 28 19:30:12.046003 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:12.045973    2572 generic.go:356] "Generic (PLEG): container finished" podID="2acc75c7-ac70-4ca4-b588-dde7608024c3" containerID="ec9d23f84ef478b1901bf4be923305355029c7e046823a4fe4472def646b2cba" exitCode=0
Jul 28 19:30:12.046494 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:12.046022    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bsh96" event={"ID":"2acc75c7-ac70-4ca4-b588-dde7608024c3","Type":"ContainerDied","Data":"ec9d23f84ef478b1901bf4be923305355029c7e046823a4fe4472def646b2cba"}
Jul 28 19:30:13.052478 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:13.052264    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bsh96" event={"ID":"2acc75c7-ac70-4ca4-b588-dde7608024c3","Type":"ContainerStarted","Data":"d5bf012d7d8207bace2e7d6d9aa46ab1a9cdecb89f073be8ab98bdacb9939504"}
Jul 28 19:30:13.075004 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:13.074960    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bsh96" podStartSLOduration=8.552921175 podStartE2EDuration="38.074946776s" totalImagesPullingTime="29.522025601s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:29:35 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:30:13.073196342 +0000 UTC m=+38.660859290" watchObservedRunningTime="2026-07-28 19:30:13.074946776 +0000 UTC m=+38.662609701"
Jul 28 19:30:13.193911 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:13.193879    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:13.248741 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:13.248693    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:13.508278 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:13.508243    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-txhmp\" (UID: \"0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:30:13.508278 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:13.508282    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:13.508483 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:13.508394    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:30:13.508483 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:13.508409    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:30:13.508483 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:13.508420    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9fc54f48b-9jtb2: secret "image-registry-tls" not found
Jul 28 19:30:13.508483 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:13.508464    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert podName:0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:17.50844667 +0000 UTC m=+43.096109601 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-txhmp" (UID: "0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5") : secret "networking-console-plugin-cert" not found
Jul 28 19:30:13.508483 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:13.508478    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls podName:f427e901-fd2a-4d35-a853-fe83fabbca35 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:17.508472431 +0000 UTC m=+43.096135354 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls") pod "image-registry-9fc54f48b-9jtb2" (UID: "f427e901-fd2a-4d35-a853-fe83fabbca35") : secret "image-registry-tls" not found
Jul 28 19:30:13.609262 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:13.609226    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:30:13.609420 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:13.609348    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:30:13.609420 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:13.609404    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls podName:eee9f5fc-fb26-4766-86a8-2b2b8b0056c5 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:17.609386961 +0000 UTC m=+43.197049885 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls") pod "dns-default-6wqlb" (UID: "eee9f5fc-fb26-4766-86a8-2b2b8b0056c5") : secret "dns-default-metrics-tls" not found
Jul 28 19:30:13.748404 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:13.748379    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:14.184473 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:14.184449    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:14.236420 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:14.236348    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:15.057532 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:15.057503    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tpngt" event={"ID":"a5c7f30b-8bac-4127-9e0a-44c45b92e47b","Type":"ContainerStarted","Data":"a2bc64ac54449291d3baaefed54c155ed4dc956a46822e0eb244c664e9e4ad27"}
Jul 28 19:30:15.073393 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:15.073355    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tpngt" podStartSLOduration=2.409764454 podStartE2EDuration="6.073341675s" totalImagesPullingTime="3.663577221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:30:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:30:15.072797212 +0000 UTC m=+40.660460159" watchObservedRunningTime="2026-07-28 19:30:15.073341675 +0000 UTC m=+40.661004653"
Jul 28 19:30:17.536556 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:17.536520    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-txhmp\" (UID: \"0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:30:17.536556 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:17.536557    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:17.536955 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:17.536682    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:30:17.536955 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:17.536703    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:30:17.536955 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:17.536715    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9fc54f48b-9jtb2: secret "image-registry-tls" not found
Jul 28 19:30:17.536955 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:17.536760    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert podName:0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:25.536743375 +0000 UTC m=+51.124406305 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-txhmp" (UID: "0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5") : secret "networking-console-plugin-cert" not found
Jul 28 19:30:17.536955 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:17.536776    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls podName:f427e901-fd2a-4d35-a853-fe83fabbca35 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:25.536770282 +0000 UTC m=+51.124433206 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls") pod "image-registry-9fc54f48b-9jtb2" (UID: "f427e901-fd2a-4d35-a853-fe83fabbca35") : secret "image-registry-tls" not found
Jul 28 19:30:17.637607 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:17.637583    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:30:17.637736 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:17.637723    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:30:17.637793 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:17.637783    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls podName:eee9f5fc-fb26-4766-86a8-2b2b8b0056c5 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:25.637766604 +0000 UTC m=+51.225429542 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls") pod "dns-default-6wqlb" (UID: "eee9f5fc-fb26-4766-86a8-2b2b8b0056c5") : secret "dns-default-metrics-tls" not found
Jul 28 19:30:20.035610 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:20.035563    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:20.135129 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:20.135097    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:20.848805 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:20.848745    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:20.948925 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:20.948900    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:21.693322 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:21.693298    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:21.748848 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:21.748819    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:24.883853 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:24.883819    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:24.981870 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:24.981843    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:25.591785 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:25.591758    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-txhmp\" (UID: \"0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:30:25.592008 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:25.591789    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:25.592008 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:25.591897    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:30:25.592008 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:25.591927    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:30:25.592008 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:25.591938    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9fc54f48b-9jtb2: secret "image-registry-tls" not found
Jul 28 19:30:25.592008 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:25.591968    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert podName:0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:41.591952451 +0000 UTC m=+67.179615376 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-txhmp" (UID: "0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5") : secret "networking-console-plugin-cert" not found
Jul 28 19:30:25.592209 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:25.592014    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls podName:f427e901-fd2a-4d35-a853-fe83fabbca35 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:41.591999783 +0000 UTC m=+67.179662707 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls") pod "image-registry-9fc54f48b-9jtb2" (UID: "f427e901-fd2a-4d35-a853-fe83fabbca35") : secret "image-registry-tls" not found
Jul 28 19:30:25.692544 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:25.692511    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:30:25.692656 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:25.692639    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:30:25.692697 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:25.692685    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls podName:eee9f5fc-fb26-4766-86a8-2b2b8b0056c5 nodeName:}" failed. No retries permitted until 2026-07-28 19:30:41.692673865 +0000 UTC m=+67.280336788 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls") pod "dns-default-6wqlb" (UID: "eee9f5fc-fb26-4766-86a8-2b2b8b0056c5") : secret "dns-default-metrics-tls" not found
Jul 28 19:30:26.809434 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:26.809401    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:26.857106 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:26.857080    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:37.036972 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:37.036944    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xf5lz"
Jul 28 19:30:37.186166 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:37.186132    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:37.285661 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:37.285635    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:40.593649 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:40.593617    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret\") pod \"global-pull-secret-syncer-vlfs6\" (UID: \"2c007a53-c536-4083-ab29-644788d631ba\") " pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:30:40.594005 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:40.593657    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs\") pod \"network-metrics-daemon-8b8d6\" (UID: \"303f736b-24a3-4e9b-8904-e4fde37ab7e0\") " pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:30:40.596974 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:40.596959    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 19:30:40.596974 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:40.596967    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 19:30:40.604045 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:40.604026    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 19:30:40.604120 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:40.604108    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs podName:303f736b-24a3-4e9b-8904-e4fde37ab7e0 nodeName:}" failed. No retries permitted until 2026-07-28 19:31:44.604088597 +0000 UTC m=+130.191751538 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs") pod "network-metrics-daemon-8b8d6" (UID: "303f736b-24a3-4e9b-8904-e4fde37ab7e0") : secret "metrics-daemon-secret" not found
Jul 28 19:30:40.607607 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:40.607582    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c007a53-c536-4083-ab29-644788d631ba-original-pull-secret\") pod \"global-pull-secret-syncer-vlfs6\" (UID: \"2c007a53-c536-4083-ab29-644788d631ba\") " pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:30:40.694454 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:40.694432    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flv9j\" (UniqueName: \"kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j\") pod \"network-check-target-cx4jl\" (UID: \"8965338e-4b6f-4d05-905f-15d353c0ea7f\") " pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:30:40.697271 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:40.697257    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 19:30:40.707592 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:40.707576    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 19:30:40.733950 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:40.733930    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flv9j\" (UniqueName: \"kubernetes.io/projected/8965338e-4b6f-4d05-905f-15d353c0ea7f-kube-api-access-flv9j\") pod \"network-check-target-cx4jl\" (UID: \"8965338e-4b6f-4d05-905f-15d353c0ea7f\") " pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:30:40.873775 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:40.873727    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlfs6"
Jul 28 19:30:40.881322 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:40.881304    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b8krh\""
Jul 28 19:30:40.889365 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:40.889350    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:30:41.038053 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:41.037980    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vlfs6"]
Jul 28 19:30:41.041554 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:30:41.041516    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c007a53_c536_4083_ab29_644788d631ba.slice/crio-0edfe4538e8d7ad209b6d53686b964312bc69bbb9a680fc2ceac67a65ee9a307 WatchSource:0}: Error finding container 0edfe4538e8d7ad209b6d53686b964312bc69bbb9a680fc2ceac67a65ee9a307: Status 404 returned error can't find the container with id 0edfe4538e8d7ad209b6d53686b964312bc69bbb9a680fc2ceac67a65ee9a307
Jul 28 19:30:41.055267 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:41.055244    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cx4jl"]
Jul 28 19:30:41.057905 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:30:41.057884    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8965338e_4b6f_4d05_905f_15d353c0ea7f.slice/crio-def20c25c911b083697285f6d45682e03e35188c03c408a35a1d9e1f01247ca7 WatchSource:0}: Error finding container def20c25c911b083697285f6d45682e03e35188c03c408a35a1d9e1f01247ca7: Status 404 returned error can't find the container with id def20c25c911b083697285f6d45682e03e35188c03c408a35a1d9e1f01247ca7
Jul 28 19:30:41.113121 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:41.113092    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cx4jl" event={"ID":"8965338e-4b6f-4d05-905f-15d353c0ea7f","Type":"ContainerStarted","Data":"def20c25c911b083697285f6d45682e03e35188c03c408a35a1d9e1f01247ca7"}
Jul 28 19:30:41.114138 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:41.114120    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vlfs6" event={"ID":"2c007a53-c536-4083-ab29-644788d631ba","Type":"ContainerStarted","Data":"0edfe4538e8d7ad209b6d53686b964312bc69bbb9a680fc2ceac67a65ee9a307"}
Jul 28 19:30:41.600924 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:41.600900    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-txhmp\" (UID: \"0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:30:41.601262 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:41.600930    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:30:41.601262 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:41.601037    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:30:41.601262 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:41.601047    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9fc54f48b-9jtb2: secret "image-registry-tls" not found
Jul 28 19:30:41.601262 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:41.601097    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls podName:f427e901-fd2a-4d35-a853-fe83fabbca35 nodeName:}" failed. No retries permitted until 2026-07-28 19:31:13.601085234 +0000 UTC m=+99.188748161 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls") pod "image-registry-9fc54f48b-9jtb2" (UID: "f427e901-fd2a-4d35-a853-fe83fabbca35") : secret "image-registry-tls" not found
Jul 28 19:30:41.601262 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:41.601037    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:30:41.601262 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:41.601155    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert podName:0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5 nodeName:}" failed. No retries permitted until 2026-07-28 19:31:13.601142619 +0000 UTC m=+99.188805543 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-txhmp" (UID: "0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5") : secret "networking-console-plugin-cert" not found
Jul 28 19:30:41.702219 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:41.702194    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:30:41.702308 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:41.702299    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:30:41.702348 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:30:41.702339    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls podName:eee9f5fc-fb26-4766-86a8-2b2b8b0056c5 nodeName:}" failed. No retries permitted until 2026-07-28 19:31:13.702327877 +0000 UTC m=+99.289990801 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls") pod "dns-default-6wqlb" (UID: "eee9f5fc-fb26-4766-86a8-2b2b8b0056c5") : secret "dns-default-metrics-tls" not found
Jul 28 19:30:42.026364 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:42.026299    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:42.126689 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:42.126655    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:46.129528 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:46.129492    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vlfs6" event={"ID":"2c007a53-c536-4083-ab29-644788d631ba","Type":"ContainerStarted","Data":"457a47fe2558f97efac4806ec7b45e963779756a1eac6c4362408cbc968f95b3"}
Jul 28 19:30:46.130977 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:46.130950    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cx4jl" event={"ID":"8965338e-4b6f-4d05-905f-15d353c0ea7f","Type":"ContainerStarted","Data":"6b8c81e78f8995d2cd5e1bbb41e016eeeada2ac10c4846bb9f1fc4ebc642df3f"}
Jul 28 19:30:46.131121 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:46.131102    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:30:46.145259 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:46.145214    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vlfs6" podStartSLOduration=66.580062231 podStartE2EDuration="1m11.14519879s" totalImagesPullingTime="4.565136559s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:29:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:30:46.144284916 +0000 UTC m=+71.731947887" watchObservedRunningTime="2026-07-28 19:30:46.14519879 +0000 UTC m=+71.732861736"
Jul 28 19:30:46.159528 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:46.159483    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cx4jl" podStartSLOduration=66.614448741 podStartE2EDuration="1m11.159472631s" totalImagesPullingTime="4.54502389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:29:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:30:46.158573049 +0000 UTC m=+71.746235995" watchObservedRunningTime="2026-07-28 19:30:46.159472631 +0000 UTC m=+71.747135623"
Jul 28 19:30:47.830770 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:47.830727    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:47.906949 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:47.906924    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:49.964116 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:49.964084    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:50.013936 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:30:50.013902    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:12.699374 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:31:12.699332    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:12.749967 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:31:12.749935    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:13.619858 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:31:13.619821    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-txhmp\" (UID: \"0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:31:13.620088 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:31:13.619860    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:31:13.620088 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:31:13.619934    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:31:13.620088 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:31:13.619995    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:31:13.620088 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:31:13.620010    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9fc54f48b-9jtb2: secret "image-registry-tls" not found
Jul 28 19:31:13.620088 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:31:13.620012    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert podName:0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5 nodeName:}" failed. No retries permitted until 2026-07-28 19:32:17.61999559 +0000 UTC m=+163.207658514 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-txhmp" (UID: "0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5") : secret "networking-console-plugin-cert" not found
Jul 28 19:31:13.620088 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:31:13.620094    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls podName:f427e901-fd2a-4d35-a853-fe83fabbca35 nodeName:}" failed. No retries permitted until 2026-07-28 19:32:17.620054936 +0000 UTC m=+163.207717860 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls") pod "image-registry-9fc54f48b-9jtb2" (UID: "f427e901-fd2a-4d35-a853-fe83fabbca35") : secret "image-registry-tls" not found
Jul 28 19:31:13.721104 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:31:13.721059    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:31:13.721422 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:31:13.721187    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:31:13.721422 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:31:13.721234    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls podName:eee9f5fc-fb26-4766-86a8-2b2b8b0056c5 nodeName:}" failed. No retries permitted until 2026-07-28 19:32:17.721220015 +0000 UTC m=+163.308882951 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls") pod "dns-default-6wqlb" (UID: "eee9f5fc-fb26-4766-86a8-2b2b8b0056c5") : secret "dns-default-metrics-tls" not found
Jul 28 19:31:17.134933 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:31:17.134904    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cx4jl"
Jul 28 19:31:44.630430 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:31:44.630390    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs\") pod \"network-metrics-daemon-8b8d6\" (UID: \"303f736b-24a3-4e9b-8904-e4fde37ab7e0\") " pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:31:44.630840 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:31:44.630516    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 19:31:44.630840 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:31:44.630580    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs podName:303f736b-24a3-4e9b-8904-e4fde37ab7e0 nodeName:}" failed. No retries permitted until 2026-07-28 19:33:46.630563673 +0000 UTC m=+252.218226597 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs") pod "network-metrics-daemon-8b8d6" (UID: "303f736b-24a3-4e9b-8904-e4fde37ab7e0") : secret "metrics-daemon-secret" not found
Jul 28 19:32:12.784628 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:32:12.784587    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp" podUID="0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5"
Jul 28 19:32:12.791765 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:32:12.791728    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2" podUID="f427e901-fd2a-4d35-a853-fe83fabbca35"
Jul 28 19:32:12.801846 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:32:12.801826    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-6wqlb" podUID="eee9f5fc-fb26-4766-86a8-2b2b8b0056c5"
Jul 28 19:32:13.330670 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:32:13.330644    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:32:13.330823 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:32:13.330645    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:32:13.330823 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:32:13.330646    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:32:13.868612 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:32:13.868577    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:32:17.655888 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:32:17.655855    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-txhmp\" (UID: \"0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:32:17.655888 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:32:17.655890    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:32:17.656328 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:32:17.655993    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:32:17.656328 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:32:17.655995    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:32:17.656328 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:32:17.656005    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9fc54f48b-9jtb2: secret "image-registry-tls" not found
Jul 28 19:32:17.656328 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:32:17.656089    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert podName:0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5 nodeName:}" failed. No retries permitted until 2026-07-28 19:34:19.656052807 +0000 UTC m=+285.243715734 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-txhmp" (UID: "0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5") : secret "networking-console-plugin-cert" not found
Jul 28 19:32:17.656328 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:32:17.656131    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls podName:f427e901-fd2a-4d35-a853-fe83fabbca35 nodeName:}" failed. No retries permitted until 2026-07-28 19:34:19.656119702 +0000 UTC m=+285.243782627 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls") pod "image-registry-9fc54f48b-9jtb2" (UID: "f427e901-fd2a-4d35-a853-fe83fabbca35") : secret "image-registry-tls" not found
Jul 28 19:32:17.756152 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:32:17.756120    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:32:17.756290 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:32:17.756251    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:32:17.756333 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:32:17.756300    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls podName:eee9f5fc-fb26-4766-86a8-2b2b8b0056c5 nodeName:}" failed. No retries permitted until 2026-07-28 19:34:19.756287373 +0000 UTC m=+285.343950298 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls") pod "dns-default-6wqlb" (UID: "eee9f5fc-fb26-4766-86a8-2b2b8b0056c5") : secret "dns-default-metrics-tls" not found
Jul 28 19:32:27.858019 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:32:27.857910    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:32:28.709903 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:32:28.709872    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:14.931036 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:14.931005    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:39.854383 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.854343    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-q8b99"]
Jul 28 19:33:39.856348 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.856322    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"]
Jul 28 19:33:39.856506 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.856450    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:39.858153 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.858130    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-jb9gx"]
Jul 28 19:33:39.858305 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.858247    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"
Jul 28 19:33:39.859739 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.859717    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx"
Jul 28 19:33:39.859879 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.859817    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 19:33:39.860399 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.860372    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 28 19:33:39.860516 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.860494    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-dpdt9\""
Jul 28 19:33:39.861262 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.861242    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 19:33:39.861262 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.861254    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 19:33:39.861445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.861381    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ldbs9\""
Jul 28 19:33:39.862461 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.862407    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 19:33:39.862461 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.862411    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 28 19:33:39.862650 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.862416    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 28 19:33:39.863882 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.863862    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:33:39.864098 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.863889    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 28 19:33:39.864098 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.863916    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-l48f6\""
Jul 28 19:33:39.864466 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.864445    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 28 19:33:39.865552 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.865494    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 28 19:33:39.865687 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.865575    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 28 19:33:39.868295 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.868271    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 28 19:33:39.870188 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.870163    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 28 19:33:39.872064 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.872029    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-q8b99"]
Jul 28 19:33:39.872976 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.872941    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"]
Jul 28 19:33:39.873978 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.873953    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-jb9gx"]
Jul 28 19:33:39.949656 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.949629    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964"]
Jul 28 19:33:39.951261 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.951247    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-ktts4"]
Jul 28 19:33:39.951376 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.951359    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964"
Jul 28 19:33:39.952821 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.952803    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-ktts4"
Jul 28 19:33:39.953732 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.953695    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 28 19:33:39.953732 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.953718    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-5rhj8\""
Jul 28 19:33:39.953959 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.953945    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 28 19:33:39.954192 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.954176    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:33:39.955816 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.955797    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-pplhf\""
Jul 28 19:33:39.966372 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.966335    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-ktts4"]
Jul 28 19:33:39.967152 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.967132    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964"]
Jul 28 19:33:39.984221 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.984197    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ln2s\" (UniqueName: \"kubernetes.io/projected/a03c223a-531b-4dce-a276-f9ef93d45d28-kube-api-access-8ln2s\") pod \"console-operator-584dcb6b54-jb9gx\" (UID: \"a03c223a-531b-4dce-a276-f9ef93d45d28\") " pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx"
Jul 28 19:33:39.984308 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.984224    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ba18d3b8-a8fb-4d09-bc33-c5213185a953-serving-cert\") pod \"insights-operator-5885fdb759-q8b99\" (UID: \"ba18d3b8-a8fb-4d09-bc33-c5213185a953\") " pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:39.984308 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.984241    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a03c223a-531b-4dce-a276-f9ef93d45d28-trusted-ca\") pod \"console-operator-584dcb6b54-jb9gx\" (UID: \"a03c223a-531b-4dce-a276-f9ef93d45d28\") " pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx"
Jul 28 19:33:39.984308 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.984256    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a03c223a-531b-4dce-a276-f9ef93d45d28-serving-cert\") pod \"console-operator-584dcb6b54-jb9gx\" (UID: \"a03c223a-531b-4dce-a276-f9ef93d45d28\") " pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx"
Jul 28 19:33:39.984405 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.984327    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba18d3b8-a8fb-4d09-bc33-c5213185a953-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-q8b99\" (UID: \"ba18d3b8-a8fb-4d09-bc33-c5213185a953\") " pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:39.984405 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.984352    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba18d3b8-a8fb-4d09-bc33-c5213185a953-service-ca-bundle\") pod \"insights-operator-5885fdb759-q8b99\" (UID: \"ba18d3b8-a8fb-4d09-bc33-c5213185a953\") " pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:39.984405 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.984391    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-phhgm\" (UID: \"d78b21b7-251a-4508-a29b-8fd37b47b31c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"
Jul 28 19:33:39.984496 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.984408    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d78b21b7-251a-4508-a29b-8fd37b47b31c-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-phhgm\" (UID: \"d78b21b7-251a-4508-a29b-8fd37b47b31c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"
Jul 28 19:33:39.984496 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.984465    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ba18d3b8-a8fb-4d09-bc33-c5213185a953-snapshots\") pod \"insights-operator-5885fdb759-q8b99\" (UID: \"ba18d3b8-a8fb-4d09-bc33-c5213185a953\") " pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:39.984555 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.984494    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc7jn\" (UniqueName: \"kubernetes.io/projected/ba18d3b8-a8fb-4d09-bc33-c5213185a953-kube-api-access-pc7jn\") pod \"insights-operator-5885fdb759-q8b99\" (UID: \"ba18d3b8-a8fb-4d09-bc33-c5213185a953\") " pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:39.984555 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.984515    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a03c223a-531b-4dce-a276-f9ef93d45d28-config\") pod \"console-operator-584dcb6b54-jb9gx\" (UID: \"a03c223a-531b-4dce-a276-f9ef93d45d28\") " pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx"
Jul 28 19:33:39.984555 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.984547    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bgw2\" (UniqueName: \"kubernetes.io/projected/d78b21b7-251a-4508-a29b-8fd37b47b31c-kube-api-access-6bgw2\") pod \"cluster-monitoring-operator-548b489cb8-phhgm\" (UID: \"d78b21b7-251a-4508-a29b-8fd37b47b31c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"
Jul 28 19:33:39.984645 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:39.984599    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba18d3b8-a8fb-4d09-bc33-c5213185a953-tmp\") pod \"insights-operator-5885fdb759-q8b99\" (UID: \"ba18d3b8-a8fb-4d09-bc33-c5213185a953\") " pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:40.074655 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.074637    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:40.085765 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.085740    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfwm7\" (UniqueName: \"kubernetes.io/projected/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-kube-api-access-jfwm7\") pod \"cluster-samples-operator-6748489656-6v964\" (UID: \"df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964"
Jul 28 19:33:40.085921 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.085781    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bgw2\" (UniqueName: \"kubernetes.io/projected/d78b21b7-251a-4508-a29b-8fd37b47b31c-kube-api-access-6bgw2\") pod \"cluster-monitoring-operator-548b489cb8-phhgm\" (UID: \"d78b21b7-251a-4508-a29b-8fd37b47b31c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"
Jul 28 19:33:40.085921 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.085829    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba18d3b8-a8fb-4d09-bc33-c5213185a953-tmp\") pod \"insights-operator-5885fdb759-q8b99\" (UID: \"ba18d3b8-a8fb-4d09-bc33-c5213185a953\") " pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:40.085921 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.085862    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ln2s\" (UniqueName: \"kubernetes.io/projected/a03c223a-531b-4dce-a276-f9ef93d45d28-kube-api-access-8ln2s\") pod \"console-operator-584dcb6b54-jb9gx\" (UID: \"a03c223a-531b-4dce-a276-f9ef93d45d28\") " pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx"
Jul 28 19:33:40.085921 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.085887    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ba18d3b8-a8fb-4d09-bc33-c5213185a953-serving-cert\") pod \"insights-operator-5885fdb759-q8b99\" (UID: \"ba18d3b8-a8fb-4d09-bc33-c5213185a953\") " pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:40.085921 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.085910    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a03c223a-531b-4dce-a276-f9ef93d45d28-trusted-ca\") pod \"console-operator-584dcb6b54-jb9gx\" (UID: \"a03c223a-531b-4dce-a276-f9ef93d45d28\") " pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx"
Jul 28 19:33:40.086205 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.085932    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a03c223a-531b-4dce-a276-f9ef93d45d28-serving-cert\") pod \"console-operator-584dcb6b54-jb9gx\" (UID: \"a03c223a-531b-4dce-a276-f9ef93d45d28\") " pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx"
Jul 28 19:33:40.086205 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.085969    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba18d3b8-a8fb-4d09-bc33-c5213185a953-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-q8b99\" (UID: \"ba18d3b8-a8fb-4d09-bc33-c5213185a953\") " pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:40.086205 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.086115    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba18d3b8-a8fb-4d09-bc33-c5213185a953-service-ca-bundle\") pod \"insights-operator-5885fdb759-q8b99\" (UID: \"ba18d3b8-a8fb-4d09-bc33-c5213185a953\") " pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:40.086205 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.086144    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-phhgm\" (UID: \"d78b21b7-251a-4508-a29b-8fd37b47b31c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"
Jul 28 19:33:40.086205 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.086164    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d78b21b7-251a-4508-a29b-8fd37b47b31c-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-phhgm\" (UID: \"d78b21b7-251a-4508-a29b-8fd37b47b31c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"
Jul 28 19:33:40.086205 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.086181    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba18d3b8-a8fb-4d09-bc33-c5213185a953-tmp\") pod \"insights-operator-5885fdb759-q8b99\" (UID: \"ba18d3b8-a8fb-4d09-bc33-c5213185a953\") " pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:40.086205 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.086191    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-6v964\" (UID: \"df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964"
Jul 28 19:33:40.086543 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:40.086257    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:33:40.086543 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.086256    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdj4z\" (UniqueName: \"kubernetes.io/projected/1798f95d-7f80-4318-8d74-02c5c0202249-kube-api-access-mdj4z\") pod \"network-check-source-7d88cb55d8-ktts4\" (UID: \"1798f95d-7f80-4318-8d74-02c5c0202249\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-ktts4"
Jul 28 19:33:40.086543 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.086303    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ba18d3b8-a8fb-4d09-bc33-c5213185a953-snapshots\") pod \"insights-operator-5885fdb759-q8b99\" (UID: \"ba18d3b8-a8fb-4d09-bc33-c5213185a953\") " pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:40.086543 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:40.086317    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls podName:d78b21b7-251a-4508-a29b-8fd37b47b31c nodeName:}" failed. No retries permitted until 2026-07-28 19:33:40.58630072 +0000 UTC m=+246.173963645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-phhgm" (UID: "d78b21b7-251a-4508-a29b-8fd37b47b31c") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:33:40.086543 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.086351    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc7jn\" (UniqueName: \"kubernetes.io/projected/ba18d3b8-a8fb-4d09-bc33-c5213185a953-kube-api-access-pc7jn\") pod \"insights-operator-5885fdb759-q8b99\" (UID: \"ba18d3b8-a8fb-4d09-bc33-c5213185a953\") " pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:40.086543 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.086377    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a03c223a-531b-4dce-a276-f9ef93d45d28-config\") pod \"console-operator-584dcb6b54-jb9gx\" (UID: \"a03c223a-531b-4dce-a276-f9ef93d45d28\") " pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx"
Jul 28 19:33:40.086846 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.086724    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba18d3b8-a8fb-4d09-bc33-c5213185a953-service-ca-bundle\") pod \"insights-operator-5885fdb759-q8b99\" (UID: \"ba18d3b8-a8fb-4d09-bc33-c5213185a953\") " pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:40.086905 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.086877    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ba18d3b8-a8fb-4d09-bc33-c5213185a953-snapshots\") pod \"insights-operator-5885fdb759-q8b99\" (UID: \"ba18d3b8-a8fb-4d09-bc33-c5213185a953\") " pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:40.087252 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.087228    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba18d3b8-a8fb-4d09-bc33-c5213185a953-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-q8b99\" (UID: \"ba18d3b8-a8fb-4d09-bc33-c5213185a953\") " pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:40.087401 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.087380    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a03c223a-531b-4dce-a276-f9ef93d45d28-config\") pod \"console-operator-584dcb6b54-jb9gx\" (UID: \"a03c223a-531b-4dce-a276-f9ef93d45d28\") " pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx"
Jul 28 19:33:40.087451 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.087407    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a03c223a-531b-4dce-a276-f9ef93d45d28-trusted-ca\") pod \"console-operator-584dcb6b54-jb9gx\" (UID: \"a03c223a-531b-4dce-a276-f9ef93d45d28\") " pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx"
Jul 28 19:33:40.087451 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.087407    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d78b21b7-251a-4508-a29b-8fd37b47b31c-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-phhgm\" (UID: \"d78b21b7-251a-4508-a29b-8fd37b47b31c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"
Jul 28 19:33:40.088866 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.088847    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a03c223a-531b-4dce-a276-f9ef93d45d28-serving-cert\") pod \"console-operator-584dcb6b54-jb9gx\" (UID: \"a03c223a-531b-4dce-a276-f9ef93d45d28\") " pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx"
Jul 28 19:33:40.088953 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.088896    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ba18d3b8-a8fb-4d09-bc33-c5213185a953-serving-cert\") pod \"insights-operator-5885fdb759-q8b99\" (UID: \"ba18d3b8-a8fb-4d09-bc33-c5213185a953\") " pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:40.093299 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.093280    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ln2s\" (UniqueName: \"kubernetes.io/projected/a03c223a-531b-4dce-a276-f9ef93d45d28-kube-api-access-8ln2s\") pod \"console-operator-584dcb6b54-jb9gx\" (UID: \"a03c223a-531b-4dce-a276-f9ef93d45d28\") " pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx"
Jul 28 19:33:40.093724 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.093705    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bgw2\" (UniqueName: \"kubernetes.io/projected/d78b21b7-251a-4508-a29b-8fd37b47b31c-kube-api-access-6bgw2\") pod \"cluster-monitoring-operator-548b489cb8-phhgm\" (UID: \"d78b21b7-251a-4508-a29b-8fd37b47b31c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"
Jul 28 19:33:40.094570 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.094548    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc7jn\" (UniqueName: \"kubernetes.io/projected/ba18d3b8-a8fb-4d09-bc33-c5213185a953-kube-api-access-pc7jn\") pod \"insights-operator-5885fdb759-q8b99\" (UID: \"ba18d3b8-a8fb-4d09-bc33-c5213185a953\") " pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:40.169243 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.169196    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-q8b99"
Jul 28 19:33:40.175307 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.175284    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:40.183392 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.183376    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx"
Jul 28 19:33:40.187386 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.187362    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfwm7\" (UniqueName: \"kubernetes.io/projected/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-kube-api-access-jfwm7\") pod \"cluster-samples-operator-6748489656-6v964\" (UID: \"df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964"
Jul 28 19:33:40.187540 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.187467    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-6v964\" (UID: \"df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964"
Jul 28 19:33:40.187540 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.187507    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdj4z\" (UniqueName: \"kubernetes.io/projected/1798f95d-7f80-4318-8d74-02c5c0202249-kube-api-access-mdj4z\") pod \"network-check-source-7d88cb55d8-ktts4\" (UID: \"1798f95d-7f80-4318-8d74-02c5c0202249\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-ktts4"
Jul 28 19:33:40.187791 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:40.187754    2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:33:40.187889 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:40.187829    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-samples-operator-tls podName:df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9 nodeName:}" failed. No retries permitted until 2026-07-28 19:33:40.687811942 +0000 UTC m=+246.275474868 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-samples-operator-tls") pod "cluster-samples-operator-6748489656-6v964" (UID: "df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9") : secret "samples-operator-tls" not found
Jul 28 19:33:40.196703 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.196677    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfwm7\" (UniqueName: \"kubernetes.io/projected/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-kube-api-access-jfwm7\") pod \"cluster-samples-operator-6748489656-6v964\" (UID: \"df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964"
Jul 28 19:33:40.197397 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.197374    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdj4z\" (UniqueName: \"kubernetes.io/projected/1798f95d-7f80-4318-8d74-02c5c0202249-kube-api-access-mdj4z\") pod \"network-check-source-7d88cb55d8-ktts4\" (UID: \"1798f95d-7f80-4318-8d74-02c5c0202249\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-ktts4"
Jul 28 19:33:40.265591 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.265557    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-ktts4"
Jul 28 19:33:40.297536 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.297507    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-q8b99"]
Jul 28 19:33:40.300318 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:33:40.300279    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba18d3b8_a8fb_4d09_bc33_c5213185a953.slice/crio-fd0dbf2e04af4d0ad0af1fdfb084acee067099fd004057461feb72f5f773227e WatchSource:0}: Error finding container fd0dbf2e04af4d0ad0af1fdfb084acee067099fd004057461feb72f5f773227e: Status 404 returned error can't find the container with id fd0dbf2e04af4d0ad0af1fdfb084acee067099fd004057461feb72f5f773227e
Jul 28 19:33:40.310765 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.310739    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-jb9gx"]
Jul 28 19:33:40.314893 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:33:40.314869    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda03c223a_531b_4dce_a276_f9ef93d45d28.slice/crio-471227a89bc759632fd2985a9c80b61a96b0d6ba1b071601f5b7066d1bbb9596 WatchSource:0}: Error finding container 471227a89bc759632fd2985a9c80b61a96b0d6ba1b071601f5b7066d1bbb9596: Status 404 returned error can't find the container with id 471227a89bc759632fd2985a9c80b61a96b0d6ba1b071601f5b7066d1bbb9596
Jul 28 19:33:40.384290 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.384263    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-ktts4"]
Jul 28 19:33:40.387023 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:33:40.387000    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1798f95d_7f80_4318_8d74_02c5c0202249.slice/crio-af9982a6a4b43cb776da29ba5de4f4da00a143c785a50d04f2cc8cb183ffe03a WatchSource:0}: Error finding container af9982a6a4b43cb776da29ba5de4f4da00a143c785a50d04f2cc8cb183ffe03a: Status 404 returned error can't find the container with id af9982a6a4b43cb776da29ba5de4f4da00a143c785a50d04f2cc8cb183ffe03a
Jul 28 19:33:40.534027 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.533996    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx" event={"ID":"a03c223a-531b-4dce-a276-f9ef93d45d28","Type":"ContainerStarted","Data":"471227a89bc759632fd2985a9c80b61a96b0d6ba1b071601f5b7066d1bbb9596"}
Jul 28 19:33:40.535329 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.535306    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-ktts4" event={"ID":"1798f95d-7f80-4318-8d74-02c5c0202249","Type":"ContainerStarted","Data":"15df5114881f04248f64d9ddfb93226d9e9cb966f736958105827b20247b0cc4"}
Jul 28 19:33:40.535418 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.535343    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-ktts4" event={"ID":"1798f95d-7f80-4318-8d74-02c5c0202249","Type":"ContainerStarted","Data":"af9982a6a4b43cb776da29ba5de4f4da00a143c785a50d04f2cc8cb183ffe03a"}
Jul 28 19:33:40.536372 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.536352    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-q8b99" event={"ID":"ba18d3b8-a8fb-4d09-bc33-c5213185a953","Type":"ContainerStarted","Data":"fd0dbf2e04af4d0ad0af1fdfb084acee067099fd004057461feb72f5f773227e"}
Jul 28 19:33:40.552840 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.552774    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-ktts4" podStartSLOduration=1.552760546 podStartE2EDuration="1.552760546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:33:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:33:40.551735291 +0000 UTC m=+246.139398239" watchObservedRunningTime="2026-07-28 19:33:40.552760546 +0000 UTC m=+246.140423492"
Jul 28 19:33:40.591423 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.591398    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-phhgm\" (UID: \"d78b21b7-251a-4508-a29b-8fd37b47b31c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"
Jul 28 19:33:40.591545 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:40.591528    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:33:40.591593 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:40.591581    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls podName:d78b21b7-251a-4508-a29b-8fd37b47b31c nodeName:}" failed. No retries permitted until 2026-07-28 19:33:41.591568638 +0000 UTC m=+247.179231565 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-phhgm" (UID: "d78b21b7-251a-4508-a29b-8fd37b47b31c") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:33:40.691928 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:40.691876    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-6v964\" (UID: \"df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964"
Jul 28 19:33:40.692021 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:40.692005    2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:33:40.692097 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:40.692058    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-samples-operator-tls podName:df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9 nodeName:}" failed. No retries permitted until 2026-07-28 19:33:41.692043683 +0000 UTC m=+247.279706608 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-samples-operator-tls") pod "cluster-samples-operator-6748489656-6v964" (UID: "df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9") : secret "samples-operator-tls" not found
Jul 28 19:33:41.598418 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:41.598381    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-phhgm\" (UID: \"d78b21b7-251a-4508-a29b-8fd37b47b31c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"
Jul 28 19:33:41.598799 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:41.598504    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:33:41.598799 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:41.598555    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls podName:d78b21b7-251a-4508-a29b-8fd37b47b31c nodeName:}" failed. No retries permitted until 2026-07-28 19:33:43.598542485 +0000 UTC m=+249.186205411 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-phhgm" (UID: "d78b21b7-251a-4508-a29b-8fd37b47b31c") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:33:41.699756 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:41.699723    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-6v964\" (UID: \"df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964"
Jul 28 19:33:41.699930 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:41.699872    2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:33:41.699988 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:41.699935    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-samples-operator-tls podName:df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9 nodeName:}" failed. No retries permitted until 2026-07-28 19:33:43.69991739 +0000 UTC m=+249.287580320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-samples-operator-tls") pod "cluster-samples-operator-6748489656-6v964" (UID: "df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9") : secret "samples-operator-tls" not found
Jul 28 19:33:42.477937 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:42.477900    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:42.578540 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:42.578509    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:43.477515 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.477485    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7gmsv"]
Jul 28 19:33:43.479577 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.479560    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:43.483323 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.483303    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 19:33:43.485253 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.485232    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 19:33:43.485400 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.485381    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9sbqm\""
Jul 28 19:33:43.500136 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.500110    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7gmsv"]
Jul 28 19:33:43.539013 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.538993    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:43.547117 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.547100    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/0.log"
Jul 28 19:33:43.547204 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.547136    2572 generic.go:356] "Generic (PLEG): container finished" podID="a03c223a-531b-4dce-a276-f9ef93d45d28" containerID="125bf0ed5dd79eda9b3a65cb9969f0e97c41b60ba0d306c6850ee5185ad4c8d5" exitCode=255
Jul 28 19:33:43.547255 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.547221    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx" event={"ID":"a03c223a-531b-4dce-a276-f9ef93d45d28","Type":"ContainerDied","Data":"125bf0ed5dd79eda9b3a65cb9969f0e97c41b60ba0d306c6850ee5185ad4c8d5"}
Jul 28 19:33:43.547430 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.547413    2572 scope.go:117] "RemoveContainer" containerID="125bf0ed5dd79eda9b3a65cb9969f0e97c41b60ba0d306c6850ee5185ad4c8d5"
Jul 28 19:33:43.548699 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.548671    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-q8b99" event={"ID":"ba18d3b8-a8fb-4d09-bc33-c5213185a953","Type":"ContainerStarted","Data":"76e77dd6204e56746386cc36f1d176073c93a0d7541b75aac349f74ad72db11c"}
Jul 28 19:33:43.582511 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.582459    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-q8b99" podStartSLOduration=2.195560222 podStartE2EDuration="4.58244585s" totalImagesPullingTime="2.386885628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:33:43.58174666 +0000 UTC m=+249.169409601" watchObservedRunningTime="2026-07-28 19:33:43.58244585 +0000 UTC m=+249.170108796"
Jul 28 19:33:43.604253 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.604228    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:43.616159 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.616138    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-phhgm\" (UID: \"d78b21b7-251a-4508-a29b-8fd37b47b31c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"
Jul 28 19:33:43.616243 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.616181    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-crio-socket\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:43.616366 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:43.616257    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:33:43.616366 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.616265    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-data-volume\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:43.616366 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.616300    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:43.616508 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:43.616379    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls podName:d78b21b7-251a-4508-a29b-8fd37b47b31c nodeName:}" failed. No retries permitted until 2026-07-28 19:33:47.616359793 +0000 UTC m=+253.204022718 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-phhgm" (UID: "d78b21b7-251a-4508-a29b-8fd37b47b31c") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:33:43.616508 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.616403    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxkqr\" (UniqueName: \"kubernetes.io/projected/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-api-access-qxkqr\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:43.616508 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.616448    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:43.717345 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.717313    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-6v964\" (UID: \"df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964"
Jul 28 19:33:43.717442 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:43.717411    2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:33:43.717511 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:43.717467    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-samples-operator-tls podName:df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9 nodeName:}" failed. No retries permitted until 2026-07-28 19:33:47.717449881 +0000 UTC m=+253.305112813 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-samples-operator-tls") pod "cluster-samples-operator-6748489656-6v964" (UID: "df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9") : secret "samples-operator-tls" not found
Jul 28 19:33:43.717911 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.717679    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-crio-socket\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:43.717911 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.717753    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-data-volume\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:43.717911 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.717781    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:43.717911 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.717800    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-crio-socket\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:43.717911 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.717877    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxkqr\" (UniqueName: \"kubernetes.io/projected/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-api-access-qxkqr\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:43.717911 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:43.717883    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:33:43.718243 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:43.717964    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls podName:c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf nodeName:}" failed. No retries permitted until 2026-07-28 19:33:44.217947091 +0000 UTC m=+249.805610017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7gmsv" (UID: "c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:33:43.718243 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.718030    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:43.718243 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.718057    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-data-volume\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:43.718243 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:43.718113    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:33:43.718243 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:43.718159    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-rbac-proxy-cm podName:c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf nodeName:}" failed. No retries permitted until 2026-07-28 19:33:44.218147196 +0000 UTC m=+249.805810120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7gmsv" (UID: "c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf") : configmap "kube-rbac-proxy" not found
Jul 28 19:33:43.727156 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:43.727138    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxkqr\" (UniqueName: \"kubernetes.io/projected/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-api-access-qxkqr\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:44.148030 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.148004    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:44.222299 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.222274    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:44.222418 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.222337    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:44.222461 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:44.222424    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:33:44.222461 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:44.222432    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:33:44.222517 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:44.222476    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-rbac-proxy-cm podName:c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf nodeName:}" failed. No retries permitted until 2026-07-28 19:33:45.222461932 +0000 UTC m=+250.810124860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7gmsv" (UID: "c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf") : configmap "kube-rbac-proxy" not found
Jul 28 19:33:44.222517 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:44.222499    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls podName:c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf nodeName:}" failed. No retries permitted until 2026-07-28 19:33:45.222488997 +0000 UTC m=+250.810151928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7gmsv" (UID: "c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:33:44.245269 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.245244    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:44.274460 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.274438    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-tmtk9"]
Jul 28 19:33:44.276429 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.276414    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tmtk9"
Jul 28 19:33:44.278938 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.278906    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 28 19:33:44.280005 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.279986    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-vjhqr\""
Jul 28 19:33:44.280005 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.280002    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 28 19:33:44.284893 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.284872    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-tmtk9"]
Jul 28 19:33:44.344768 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.344740    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:44.424898 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.424849    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sczz9\" (UniqueName: \"kubernetes.io/projected/c03a93c7-a1b6-40e3-9bb3-93081ef40fa3-kube-api-access-sczz9\") pod \"migrator-6865c9967-tmtk9\" (UID: \"c03a93c7-a1b6-40e3-9bb3-93081ef40fa3\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tmtk9"
Jul 28 19:33:44.444902 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.444879    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:44.525970 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.525943    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sczz9\" (UniqueName: \"kubernetes.io/projected/c03a93c7-a1b6-40e3-9bb3-93081ef40fa3-kube-api-access-sczz9\") pod \"migrator-6865c9967-tmtk9\" (UID: \"c03a93c7-a1b6-40e3-9bb3-93081ef40fa3\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tmtk9"
Jul 28 19:33:44.534013 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.533990    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sczz9\" (UniqueName: \"kubernetes.io/projected/c03a93c7-a1b6-40e3-9bb3-93081ef40fa3-kube-api-access-sczz9\") pod \"migrator-6865c9967-tmtk9\" (UID: \"c03a93c7-a1b6-40e3-9bb3-93081ef40fa3\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tmtk9"
Jul 28 19:33:44.553418 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.553396    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 19:33:44.553804 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.553789    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/0.log"
Jul 28 19:33:44.553869 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.553824    2572 generic.go:356] "Generic (PLEG): container finished" podID="a03c223a-531b-4dce-a276-f9ef93d45d28" containerID="31d22a0b2c10dc19fd332e42041938e6acc8ced6364f452daf211b8856bbd208" exitCode=255
Jul 28 19:33:44.553911 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.553892    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx" event={"ID":"a03c223a-531b-4dce-a276-f9ef93d45d28","Type":"ContainerDied","Data":"31d22a0b2c10dc19fd332e42041938e6acc8ced6364f452daf211b8856bbd208"}
Jul 28 19:33:44.553958 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.553947    2572 scope.go:117] "RemoveContainer" containerID="125bf0ed5dd79eda9b3a65cb9969f0e97c41b60ba0d306c6850ee5185ad4c8d5"
Jul 28 19:33:44.554254 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.554224    2572 scope.go:117] "RemoveContainer" containerID="31d22a0b2c10dc19fd332e42041938e6acc8ced6364f452daf211b8856bbd208"
Jul 28 19:33:44.554393 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:44.554377    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-jb9gx_openshift-console-operator(a03c223a-531b-4dce-a276-f9ef93d45d28)\"" pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx" podUID="a03c223a-531b-4dce-a276-f9ef93d45d28"
Jul 28 19:33:44.585284 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.585263    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tmtk9"
Jul 28 19:33:44.704411 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.704232    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-tmtk9"]
Jul 28 19:33:44.707000 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:33:44.706967    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc03a93c7_a1b6_40e3_9bb3_93081ef40fa3.slice/crio-64cab3bd76b851a0e7d6dcb3224a44c7364cdee1091d6bdb2703f044657e2693 WatchSource:0}: Error finding container 64cab3bd76b851a0e7d6dcb3224a44c7364cdee1091d6bdb2703f044657e2693: Status 404 returned error can't find the container with id 64cab3bd76b851a0e7d6dcb3224a44c7364cdee1091d6bdb2703f044657e2693
Jul 28 19:33:44.846243 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.846212    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:44.947163 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:44.947138    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:45.232330 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:45.232302    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:45.232540 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:45.232378    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:45.232540 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:45.232436    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:33:45.232540 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:45.232485    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:33:45.232540 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:45.232509    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-rbac-proxy-cm podName:c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf nodeName:}" failed. No retries permitted until 2026-07-28 19:33:47.232487648 +0000 UTC m=+252.820150587 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7gmsv" (UID: "c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf") : configmap "kube-rbac-proxy" not found
Jul 28 19:33:45.232753 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:45.232548    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls podName:c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf nodeName:}" failed. No retries permitted until 2026-07-28 19:33:47.232532996 +0000 UTC m=+252.820195935 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7gmsv" (UID: "c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:33:45.558052 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:45.558027    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 19:33:45.558488 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:45.558343    2572 scope.go:117] "RemoveContainer" containerID="31d22a0b2c10dc19fd332e42041938e6acc8ced6364f452daf211b8856bbd208"
Jul 28 19:33:45.558549 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:45.558503    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-jb9gx_openshift-console-operator(a03c223a-531b-4dce-a276-f9ef93d45d28)\"" pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx" podUID="a03c223a-531b-4dce-a276-f9ef93d45d28"
Jul 28 19:33:45.559293 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:45.559272    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tmtk9" event={"ID":"c03a93c7-a1b6-40e3-9bb3-93081ef40fa3","Type":"ContainerStarted","Data":"64cab3bd76b851a0e7d6dcb3224a44c7364cdee1091d6bdb2703f044657e2693"}
Jul 28 19:33:45.672782 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:45.672762    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-td8fx_4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5/dns-node-resolver/0.log"
Jul 28 19:33:46.444060 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.444034    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-sfjmp"]
Jul 28 19:33:46.447033 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.447019    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-sfjmp"
Jul 28 19:33:46.449384 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.449363    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 28 19:33:46.449476 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.449421    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 28 19:33:46.450475 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.450458    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-k99wd\""
Jul 28 19:33:46.450475 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.450458    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 28 19:33:46.450606 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.450501    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 28 19:33:46.456108 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.456086    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-sfjmp"]
Jul 28 19:33:46.482168 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.482147    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:46.543259 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.543236    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b53731f2-0a98-4ddd-a0d9-ab155762dbb3-signing-cabundle\") pod \"service-ca-658d7b7c78-sfjmp\" (UID: \"b53731f2-0a98-4ddd-a0d9-ab155762dbb3\") " pod="openshift-service-ca/service-ca-658d7b7c78-sfjmp"
Jul 28 19:33:46.543338 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.543265    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hgkm\" (UniqueName: \"kubernetes.io/projected/b53731f2-0a98-4ddd-a0d9-ab155762dbb3-kube-api-access-8hgkm\") pod \"service-ca-658d7b7c78-sfjmp\" (UID: \"b53731f2-0a98-4ddd-a0d9-ab155762dbb3\") " pod="openshift-service-ca/service-ca-658d7b7c78-sfjmp"
Jul 28 19:33:46.543338 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.543327    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b53731f2-0a98-4ddd-a0d9-ab155762dbb3-signing-key\") pod \"service-ca-658d7b7c78-sfjmp\" (UID: \"b53731f2-0a98-4ddd-a0d9-ab155762dbb3\") " pod="openshift-service-ca/service-ca-658d7b7c78-sfjmp"
Jul 28 19:33:46.564806 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.564785    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tmtk9" event={"ID":"c03a93c7-a1b6-40e3-9bb3-93081ef40fa3","Type":"ContainerStarted","Data":"d386b651cf0546312a51560b01301369b68689fe4e08851352a167f3ca6f3a3f"}
Jul 28 19:33:46.565181 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.564822    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tmtk9" event={"ID":"c03a93c7-a1b6-40e3-9bb3-93081ef40fa3","Type":"ContainerStarted","Data":"bd7e6d06e4cb7fb4d29d202954ec727f5f49a8efd68ca08e03e6e84cdbc0d8c9"}
Jul 28 19:33:46.579988 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.579952    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tmtk9" podStartSLOduration=1.384400529 podStartE2EDuration="2.579940253s" totalImagesPullingTime="1.195539724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:33:46.579292851 +0000 UTC m=+252.166955800" watchObservedRunningTime="2026-07-28 19:33:46.579940253 +0000 UTC m=+252.167603199"
Jul 28 19:33:46.644439 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.644418    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b53731f2-0a98-4ddd-a0d9-ab155762dbb3-signing-cabundle\") pod \"service-ca-658d7b7c78-sfjmp\" (UID: \"b53731f2-0a98-4ddd-a0d9-ab155762dbb3\") " pod="openshift-service-ca/service-ca-658d7b7c78-sfjmp"
Jul 28 19:33:46.644502 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.644446    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hgkm\" (UniqueName: \"kubernetes.io/projected/b53731f2-0a98-4ddd-a0d9-ab155762dbb3-kube-api-access-8hgkm\") pod \"service-ca-658d7b7c78-sfjmp\" (UID: \"b53731f2-0a98-4ddd-a0d9-ab155762dbb3\") " pod="openshift-service-ca/service-ca-658d7b7c78-sfjmp"
Jul 28 19:33:46.644502 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.644463    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs\") pod \"network-metrics-daemon-8b8d6\" (UID: \"303f736b-24a3-4e9b-8904-e4fde37ab7e0\") " pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:33:46.644568 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.644515    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b53731f2-0a98-4ddd-a0d9-ab155762dbb3-signing-key\") pod \"service-ca-658d7b7c78-sfjmp\" (UID: \"b53731f2-0a98-4ddd-a0d9-ab155762dbb3\") " pod="openshift-service-ca/service-ca-658d7b7c78-sfjmp"
Jul 28 19:33:46.644615 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:46.644598    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 19:33:46.644656 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:46.644650    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs podName:303f736b-24a3-4e9b-8904-e4fde37ab7e0 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:48.64463607 +0000 UTC m=+374.232298994 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs") pod "network-metrics-daemon-8b8d6" (UID: "303f736b-24a3-4e9b-8904-e4fde37ab7e0") : secret "metrics-daemon-secret" not found
Jul 28 19:33:46.645144 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.645126    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b53731f2-0a98-4ddd-a0d9-ab155762dbb3-signing-cabundle\") pod \"service-ca-658d7b7c78-sfjmp\" (UID: \"b53731f2-0a98-4ddd-a0d9-ab155762dbb3\") " pod="openshift-service-ca/service-ca-658d7b7c78-sfjmp"
Jul 28 19:33:46.646803 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.646783    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b53731f2-0a98-4ddd-a0d9-ab155762dbb3-signing-key\") pod \"service-ca-658d7b7c78-sfjmp\" (UID: \"b53731f2-0a98-4ddd-a0d9-ab155762dbb3\") " pod="openshift-service-ca/service-ca-658d7b7c78-sfjmp"
Jul 28 19:33:46.652657 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.652636    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hgkm\" (UniqueName: \"kubernetes.io/projected/b53731f2-0a98-4ddd-a0d9-ab155762dbb3-kube-api-access-8hgkm\") pod \"service-ca-658d7b7c78-sfjmp\" (UID: \"b53731f2-0a98-4ddd-a0d9-ab155762dbb3\") " pod="openshift-service-ca/service-ca-658d7b7c78-sfjmp"
Jul 28 19:33:46.656709 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.656684    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:46.671466 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.671442    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tgq2d_be4dd071-8c56-455a-bb1e-440b5b292f3b/node-ca/0.log"
Jul 28 19:33:46.754850 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.754797    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-sfjmp"
Jul 28 19:33:46.758277 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.758255    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:46.872118 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:46.872097    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-sfjmp"]
Jul 28 19:33:46.874304 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:33:46.874279    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb53731f2_0a98_4ddd_a0d9_ab155762dbb3.slice/crio-8c9c2b0cb7c4b518943a37cc0edf12273fe0372868e6ec0e68d09c72eba9525d WatchSource:0}: Error finding container 8c9c2b0cb7c4b518943a37cc0edf12273fe0372868e6ec0e68d09c72eba9525d: Status 404 returned error can't find the container with id 8c9c2b0cb7c4b518943a37cc0edf12273fe0372868e6ec0e68d09c72eba9525d
Jul 28 19:33:47.248990 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:47.248944    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:47.249168 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:47.249039    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:47.249168 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:47.249061    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:33:47.249168 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:47.249133    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-rbac-proxy-cm podName:c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf nodeName:}" failed. No retries permitted until 2026-07-28 19:33:51.249118614 +0000 UTC m=+256.836781538 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7gmsv" (UID: "c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf") : configmap "kube-rbac-proxy" not found
Jul 28 19:33:47.249168 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:47.249151    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:33:47.249309 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:47.249190    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls podName:c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf nodeName:}" failed. No retries permitted until 2026-07-28 19:33:51.249179245 +0000 UTC m=+256.836842174 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7gmsv" (UID: "c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:33:47.404578 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:47.404545    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:47.472388 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:47.472362    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tpngt_a5c7f30b-8bac-4127-9e0a-44c45b92e47b/serve-healthcheck-canary/0.log"
Jul 28 19:33:47.569858 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:47.569824    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-sfjmp" event={"ID":"b53731f2-0a98-4ddd-a0d9-ab155762dbb3","Type":"ContainerStarted","Data":"8c9c2b0cb7c4b518943a37cc0edf12273fe0372868e6ec0e68d09c72eba9525d"}
Jul 28 19:33:47.652697 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:47.652647    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-phhgm\" (UID: \"d78b21b7-251a-4508-a29b-8fd37b47b31c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"
Jul 28 19:33:47.652874 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:47.652772    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:33:47.652874 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:47.652850    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls podName:d78b21b7-251a-4508-a29b-8fd37b47b31c nodeName:}" failed. No retries permitted until 2026-07-28 19:33:55.65282872 +0000 UTC m=+261.240491658 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-phhgm" (UID: "d78b21b7-251a-4508-a29b-8fd37b47b31c") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:33:47.753807 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:47.753778    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-6v964\" (UID: \"df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964"
Jul 28 19:33:47.753953 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:47.753935    2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:33:47.754001 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:47.753994    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-samples-operator-tls podName:df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9 nodeName:}" failed. No retries permitted until 2026-07-28 19:33:55.753979331 +0000 UTC m=+261.341642255 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-samples-operator-tls") pod "cluster-samples-operator-6748489656-6v964" (UID: "df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9") : secret "samples-operator-tls" not found
Jul 28 19:33:49.579149 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:49.579112    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-sfjmp" event={"ID":"b53731f2-0a98-4ddd-a0d9-ab155762dbb3","Type":"ContainerStarted","Data":"fc6ebb28e5de24b1cc8f535e0a368e8fa9806fa883b99faacfa8de4a5ad03f1b"}
Jul 28 19:33:49.594909 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:49.594869    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-sfjmp" podStartSLOduration=1.797189644 podStartE2EDuration="3.59485556s" totalImagesPullingTime="1.797665916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:33:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:33:49.593658171 +0000 UTC m=+255.181321111" watchObservedRunningTime="2026-07-28 19:33:49.59485556 +0000 UTC m=+255.182518505"
Jul 28 19:33:50.077249 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:50.077217    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:50.184214 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:50.184140    2572 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx"
Jul 28 19:33:50.184214 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:50.184198    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx"
Jul 28 19:33:50.184544 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:50.184526    2572 scope.go:117] "RemoveContainer" containerID="31d22a0b2c10dc19fd332e42041938e6acc8ced6364f452daf211b8856bbd208"
Jul 28 19:33:50.184715 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:50.184695    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-jb9gx_openshift-console-operator(a03c223a-531b-4dce-a276-f9ef93d45d28)\"" pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx" podUID="a03c223a-531b-4dce-a276-f9ef93d45d28"
Jul 28 19:33:50.225285 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:50.225241    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:50.326906 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:50.326877    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:50.428275 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:50.428252    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:50.538933 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:50.538870    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:51.173829 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:51.173793    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:51.281594 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:51.281561    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:51.281731 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:51.281633    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:51.281731 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:51.281681    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:33:51.281731 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:51.281721    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:33:51.281859 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:51.281746    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-rbac-proxy-cm podName:c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf nodeName:}" failed. No retries permitted until 2026-07-28 19:33:59.281728181 +0000 UTC m=+264.869391113 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7gmsv" (UID: "c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf") : configmap "kube-rbac-proxy" not found
Jul 28 19:33:51.281859 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:51.281765    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls podName:c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf nodeName:}" failed. No retries permitted until 2026-07-28 19:33:59.281755153 +0000 UTC m=+264.869418079 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7gmsv" (UID: "c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:33:51.369598 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:51.369562    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:51.965549 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:51.965514    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:52.565844 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:52.565811    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:52.763379 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:52.763347    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:53.161422 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:53.161393    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:53.369694 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:53.369669    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:53.590180 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:53.590135    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:53.962476 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:53.962384    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:54.014655 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:54.014625    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:54.562992 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:54.562964    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:54.766878 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:54.766846    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:54.964755 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:54.964678    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:55.365775 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:55.365749    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:55.712749 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:55.712686    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-phhgm\" (UID: \"d78b21b7-251a-4508-a29b-8fd37b47b31c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"
Jul 28 19:33:55.712873 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:55.712860    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:33:55.712953 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:55.712941    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls podName:d78b21b7-251a-4508-a29b-8fd37b47b31c nodeName:}" failed. No retries permitted until 2026-07-28 19:34:11.712918963 +0000 UTC m=+277.300581890 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-phhgm" (UID: "d78b21b7-251a-4508-a29b-8fd37b47b31c") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:33:55.813405 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:55.813382    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-6v964\" (UID: \"df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964"
Jul 28 19:33:55.815650 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:55.815628    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-6v964\" (UID: \"df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964"
Jul 28 19:33:55.860716 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:55.860696    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964"
Jul 28 19:33:55.980682 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:55.980617    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964"]
Jul 28 19:33:56.170033 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:56.170007    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:56.478835 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:56.478800    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:56.563815 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:56.563792    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:56.602540 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:56.602515    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964" event={"ID":"df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9","Type":"ContainerStarted","Data":"ac41f3c13171c4bb1ded4230f664b7e6d0066fbfcdd62221ab18712d7fc140db"}
Jul 28 19:33:56.771368 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:56.771263    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:56.821505 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:56.821480    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:56.967393 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:56.967365    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:57.367804 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:57.367766    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:57.969580 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:57.969514    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:58.362123 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:58.362086    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:58.610548 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:58.610515    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964" event={"ID":"df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9","Type":"ContainerStarted","Data":"5a518533ef7f0f9549634c921494900b46cfa31cc8e0e78d95ce476698236375"}
Jul 28 19:33:58.610682 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:58.610597    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964" event={"ID":"df5f5d24-c088-4c66-8edb-c6ccfbbf0cf9","Type":"ContainerStarted","Data":"527f13b96bd6d2e1ab95a9cd1fe9b69d58001528c93748416ce045d40886b9bf"}
Jul 28 19:33:58.625834 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:58.625762    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6v964" podStartSLOduration=17.957768106 podStartE2EDuration="19.625745991s" totalImagesPullingTime="1.667977885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:33:58.624818717 +0000 UTC m=+264.212481663" watchObservedRunningTime="2026-07-28 19:33:58.625745991 +0000 UTC m=+264.213408937"
Jul 28 19:33:58.762366 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:58.762337    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:59.164388 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:59.164361    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:59.342313 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:59.342281    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:59.342504 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:59.342379    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:59.342504 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:59.342440    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:33:59.342619 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:33:59.342514    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls podName:c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf nodeName:}" failed. No retries permitted until 2026-07-28 19:34:15.342494113 +0000 UTC m=+280.930157055 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7gmsv" (UID: "c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:33:59.342860 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:59.342839    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:33:59.559586 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:59.559561    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:59.764151 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:33:59.764110    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:00.168082 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:00.168032    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:00.565337 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:00.565297    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:00.771276 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:00.771249    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:05.858251 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:05.858223    2572 scope.go:117] "RemoveContainer" containerID="31d22a0b2c10dc19fd332e42041938e6acc8ced6364f452daf211b8856bbd208"
Jul 28 19:34:06.351502 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:06.351464    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:06.403099 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:06.403053    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:06.628468 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:06.628388    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:06.637245 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:06.637223    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 19:34:06.637342 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:06.637297    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx" event={"ID":"a03c223a-531b-4dce-a276-f9ef93d45d28","Type":"ContainerStarted","Data":"345ba03a73a04d718ffec52ca8ff4ce31a6e9473cdc976d5b62469e7f59be92a"}
Jul 28 19:34:06.637595 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:06.637576    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx"
Jul 28 19:34:06.642036 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:06.642018    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx"
Jul 28 19:34:06.657716 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:06.657680    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-jb9gx" podStartSLOduration=25.284531873 podStartE2EDuration="27.65766897s" totalImagesPullingTime="2.373137097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:34:06.656712716 +0000 UTC m=+272.244375689" watchObservedRunningTime="2026-07-28 19:34:06.65766897 +0000 UTC m=+272.245331916"
Jul 28 19:34:06.681804 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:06.681783    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:06.736631 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:06.736603    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:06.939983 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:06.939914    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:07.309046 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:07.309022    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:07.406439 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:07.406414    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:07.506152 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:07.506131    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:07.928219 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:07.928180    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:07.981381 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:07.981356    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:08.130229 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:08.130203    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:08.518135 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:08.518098    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:08.718048 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:08.718016    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:08.853234 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:08.853203    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:10.333371 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:10.333335    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:10.389168 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:10.389118    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:11.735446 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:11.735420    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-phhgm\" (UID: \"d78b21b7-251a-4508-a29b-8fd37b47b31c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"
Jul 28 19:34:11.737704 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:11.737682    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d78b21b7-251a-4508-a29b-8fd37b47b31c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-phhgm\" (UID: \"d78b21b7-251a-4508-a29b-8fd37b47b31c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"
Jul 28 19:34:11.978292 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:11.978265    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"
Jul 28 19:34:12.136540 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:12.136517    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm"]
Jul 28 19:34:12.141195 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:34:12.139167    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd78b21b7_251a_4508_a29b_8fd37b47b31c.slice/crio-eb8f2ac0c11724790a708f736033ab20dfea5585b7d4f9794920c7de66d4a0f2 WatchSource:0}: Error finding container eb8f2ac0c11724790a708f736033ab20dfea5585b7d4f9794920c7de66d4a0f2: Status 404 returned error can't find the container with id eb8f2ac0c11724790a708f736033ab20dfea5585b7d4f9794920c7de66d4a0f2
Jul 28 19:34:12.656425 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:12.656394    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm" event={"ID":"d78b21b7-251a-4508-a29b-8fd37b47b31c","Type":"ContainerStarted","Data":"eb8f2ac0c11724790a708f736033ab20dfea5585b7d4f9794920c7de66d4a0f2"}
Jul 28 19:34:14.468032 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:14.467957    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:14.566133 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:14.566058    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:14.642572 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:14.642541    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:14.664732 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:14.664701    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm" event={"ID":"d78b21b7-251a-4508-a29b-8fd37b47b31c","Type":"ContainerStarted","Data":"3c9582d3c35d33c847922a7c2a75d33a8269d76e3e3f286d2d47aa079136b26c"}
Jul 28 19:34:14.683002 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:14.682958    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phhgm" podStartSLOduration=33.688213858 podStartE2EDuration="35.682943487s" totalImagesPullingTime="1.994729629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:34:14.68261617 +0000 UTC m=+280.270279141" watchObservedRunningTime="2026-07-28 19:34:14.682943487 +0000 UTC m=+280.270606433"
Jul 28 19:34:14.930977 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:14.930955    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:15.149974 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.149949    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69cdff8dd4-d9l29"]
Jul 28 19:34:15.151661 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.151645    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.154766 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.154751    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 19:34:15.155951 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.155874    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hlt9x\""
Jul 28 19:34:15.155951 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.155891    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 19:34:15.155951 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.155907    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 19:34:15.155951 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.155927    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 19:34:15.156209 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.155925    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 19:34:15.156209 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.156021    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 19:34:15.156209 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.156084    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 19:34:15.165380 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.165363    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69cdff8dd4-d9l29"]
Jul 28 19:34:15.265105 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.265037    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb6decc4-64ba-4211-b929-44e406c4dfca-service-ca\") pod \"console-69cdff8dd4-d9l29\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") " pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.265186 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.265106    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb6decc4-64ba-4211-b929-44e406c4dfca-console-config\") pod \"console-69cdff8dd4-d9l29\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") " pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.265186 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.265153    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb6decc4-64ba-4211-b929-44e406c4dfca-console-serving-cert\") pod \"console-69cdff8dd4-d9l29\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") " pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.265255 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.265186    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb6decc4-64ba-4211-b929-44e406c4dfca-oauth-serving-cert\") pod \"console-69cdff8dd4-d9l29\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") " pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.265255 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.265209    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7vgw\" (UniqueName: \"kubernetes.io/projected/cb6decc4-64ba-4211-b929-44e406c4dfca-kube-api-access-t7vgw\") pod \"console-69cdff8dd4-d9l29\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") " pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.265255 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.265251    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb6decc4-64ba-4211-b929-44e406c4dfca-console-oauth-config\") pod \"console-69cdff8dd4-d9l29\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") " pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.365602 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.365578    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb6decc4-64ba-4211-b929-44e406c4dfca-console-oauth-config\") pod \"console-69cdff8dd4-d9l29\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") " pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.365692 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.365617    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb6decc4-64ba-4211-b929-44e406c4dfca-service-ca\") pod \"console-69cdff8dd4-d9l29\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") " pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.365692 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.365640    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:34:15.365879 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.365859    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb6decc4-64ba-4211-b929-44e406c4dfca-console-config\") pod \"console-69cdff8dd4-d9l29\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") " pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.365942 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.365893    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb6decc4-64ba-4211-b929-44e406c4dfca-console-serving-cert\") pod \"console-69cdff8dd4-d9l29\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") " pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.366129 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.366100    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb6decc4-64ba-4211-b929-44e406c4dfca-oauth-serving-cert\") pod \"console-69cdff8dd4-d9l29\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") " pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.366240 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.366154    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7vgw\" (UniqueName: \"kubernetes.io/projected/cb6decc4-64ba-4211-b929-44e406c4dfca-kube-api-access-t7vgw\") pod \"console-69cdff8dd4-d9l29\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") " pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.366446 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.366424    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb6decc4-64ba-4211-b929-44e406c4dfca-service-ca\") pod \"console-69cdff8dd4-d9l29\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") " pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.366576 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.366558    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb6decc4-64ba-4211-b929-44e406c4dfca-console-config\") pod \"console-69cdff8dd4-d9l29\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") " pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.366803 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.366781    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb6decc4-64ba-4211-b929-44e406c4dfca-oauth-serving-cert\") pod \"console-69cdff8dd4-d9l29\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") " pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.368106 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.368062    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7gmsv\" (UID: \"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf\") " pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:34:15.368213 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.368197    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb6decc4-64ba-4211-b929-44e406c4dfca-console-oauth-config\") pod \"console-69cdff8dd4-d9l29\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") " pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.368255 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.368225    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb6decc4-64ba-4211-b929-44e406c4dfca-console-serving-cert\") pod \"console-69cdff8dd4-d9l29\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") " pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.373741 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.373701    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7vgw\" (UniqueName: \"kubernetes.io/projected/cb6decc4-64ba-4211-b929-44e406c4dfca-kube-api-access-t7vgw\") pod \"console-69cdff8dd4-d9l29\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") " pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.460245 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.460226    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:15.579833 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.579805    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69cdff8dd4-d9l29"]
Jul 28 19:34:15.582845 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:34:15.582821    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb6decc4_64ba_4211_b929_44e406c4dfca.slice/crio-fdd3e1548aa018496e381496fa666a3ccea2182ce04bbf0f371f0bd914a3b78a WatchSource:0}: Error finding container fdd3e1548aa018496e381496fa666a3ccea2182ce04bbf0f371f0bd914a3b78a: Status 404 returned error can't find the container with id fdd3e1548aa018496e381496fa666a3ccea2182ce04bbf0f371f0bd914a3b78a
Jul 28 19:34:15.588889 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.588870    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7gmsv"
Jul 28 19:34:15.669274 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.669243    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69cdff8dd4-d9l29" event={"ID":"cb6decc4-64ba-4211-b929-44e406c4dfca","Type":"ContainerStarted","Data":"fdd3e1548aa018496e381496fa666a3ccea2182ce04bbf0f371f0bd914a3b78a"}
Jul 28 19:34:15.732083 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:15.731942    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7gmsv"]
Jul 28 19:34:15.734500 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:34:15.734472    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc60fd0f2_b6d3_450b_9a2e_6a2ca8281ccf.slice/crio-f910c560c97f382c1e56dcf405f345c50c7d7597f4466f538a02350d90330362 WatchSource:0}: Error finding container f910c560c97f382c1e56dcf405f345c50c7d7597f4466f538a02350d90330362: Status 404 returned error can't find the container with id f910c560c97f382c1e56dcf405f345c50c7d7597f4466f538a02350d90330362
Jul 28 19:34:16.331416 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:34:16.331368    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-6wqlb" podUID="eee9f5fc-fb26-4766-86a8-2b2b8b0056c5"
Jul 28 19:34:16.331923 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:34:16.331606    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp" podUID="0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5"
Jul 28 19:34:16.331923 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:34:16.331814    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2" podUID="f427e901-fd2a-4d35-a853-fe83fabbca35"
Jul 28 19:34:16.676676 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:16.675928    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:34:16.676676 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:16.676270    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7gmsv" event={"ID":"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf","Type":"ContainerStarted","Data":"f910c560c97f382c1e56dcf405f345c50c7d7597f4466f538a02350d90330362"}
Jul 28 19:34:16.676676 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:16.676351    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:34:16.676676 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:16.676636    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:34:17.655729 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:17.655659    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:17.723591 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:17.723547    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:17.779385 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:17.779359    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:18.340209 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:18.340161    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:18.390854 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:18.390824    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:19.701276 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:19.701248    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-txhmp\" (UID: \"0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:34:19.701695 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:19.701338    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:34:19.703744 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:19.703716    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-txhmp\" (UID: \"0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:34:19.703870 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:19.703828    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls\") pod \"image-registry-9fc54f48b-9jtb2\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") " pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:34:19.801745 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:19.801704    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:34:19.804096 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:19.804057    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee9f5fc-fb26-4766-86a8-2b2b8b0056c5-metrics-tls\") pod \"dns-default-6wqlb\" (UID: \"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5\") " pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:34:19.981350 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:19.981284    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pwrtc\""
Jul 28 19:34:19.981350 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:19.981332    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-phlnm\""
Jul 28 19:34:19.981547 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:19.981296    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5qrcf\""
Jul 28 19:34:19.987598 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:19.987577    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:34:19.987736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:19.987651    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"
Jul 28 19:34:19.987736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:19.987684    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:34:21.816131 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:21.816104    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-txhmp"]
Jul 28 19:34:21.818193 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:34:21.818125    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bf17d43_9e36_4a59_8cf2_8e23f4ed98f5.slice/crio-41f9f52a45834cded8e10bfa679c4625b63a7eec9603ffcc3cdd738002104c81 WatchSource:0}: Error finding container 41f9f52a45834cded8e10bfa679c4625b63a7eec9603ffcc3cdd738002104c81: Status 404 returned error can't find the container with id 41f9f52a45834cded8e10bfa679c4625b63a7eec9603ffcc3cdd738002104c81
Jul 28 19:34:22.040134 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:22.040047    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6wqlb"]
Jul 28 19:34:22.042963 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:22.042874    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9fc54f48b-9jtb2"]
Jul 28 19:34:22.042963 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:34:22.042871    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee9f5fc_fb26_4766_86a8_2b2b8b0056c5.slice/crio-7f90a13312fee443289e13cf189828219eee1be065af9eac46d55c2a113c7d57 WatchSource:0}: Error finding container 7f90a13312fee443289e13cf189828219eee1be065af9eac46d55c2a113c7d57: Status 404 returned error can't find the container with id 7f90a13312fee443289e13cf189828219eee1be065af9eac46d55c2a113c7d57
Jul 28 19:34:22.045201 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:34:22.045176    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf427e901_fd2a_4d35_a853_fe83fabbca35.slice/crio-29e873e68f91f6f8196d9c0c90714f19e9bd69a288524dce955e3be25dfecf67 WatchSource:0}: Error finding container 29e873e68f91f6f8196d9c0c90714f19e9bd69a288524dce955e3be25dfecf67: Status 404 returned error can't find the container with id 29e873e68f91f6f8196d9c0c90714f19e9bd69a288524dce955e3be25dfecf67
Jul 28 19:34:22.703765 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:22.703729    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69cdff8dd4-d9l29" event={"ID":"cb6decc4-64ba-4211-b929-44e406c4dfca","Type":"ContainerStarted","Data":"cdd3a97e446f1ef25a264a34269b47f44b592af059031d974673d56e21ae8fec"}
Jul 28 19:34:22.706241 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:22.706186    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp" event={"ID":"0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5","Type":"ContainerStarted","Data":"41f9f52a45834cded8e10bfa679c4625b63a7eec9603ffcc3cdd738002104c81"}
Jul 28 19:34:22.708934 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:22.708868    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7gmsv" event={"ID":"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf","Type":"ContainerStarted","Data":"f21f9e3d14051842894a29ab6704f5b83d9cad2ca328fb0c1d4350020849faa5"}
Jul 28 19:34:22.711489 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:22.711467    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6wqlb" event={"ID":"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5","Type":"ContainerStarted","Data":"7f90a13312fee443289e13cf189828219eee1be065af9eac46d55c2a113c7d57"}
Jul 28 19:34:22.714976 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:22.714951    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2" event={"ID":"f427e901-fd2a-4d35-a853-fe83fabbca35","Type":"ContainerStarted","Data":"d945c6fba34b66e465cacc9d117cf1d7f29d07cf21fc38bb46674401d239acd6"}
Jul 28 19:34:22.715061 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:22.714995    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2" event={"ID":"f427e901-fd2a-4d35-a853-fe83fabbca35","Type":"ContainerStarted","Data":"29e873e68f91f6f8196d9c0c90714f19e9bd69a288524dce955e3be25dfecf67"}
Jul 28 19:34:22.715540 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:22.715521    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:34:22.723083 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:22.723026    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69cdff8dd4-d9l29" podStartSLOduration=1.646980436 podStartE2EDuration="7.723007728s" totalImagesPullingTime="6.076027292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:34:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:34:22.721153908 +0000 UTC m=+288.308816856" watchObservedRunningTime="2026-07-28 19:34:22.723007728 +0000 UTC m=+288.310670677"
Jul 28 19:34:22.739479 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:22.739342    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2" podStartSLOduration=287.73932451 podStartE2EDuration="4m47.73932451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:29:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:34:22.738698362 +0000 UTC m=+288.326361300" watchObservedRunningTime="2026-07-28 19:34:22.73932451 +0000 UTC m=+288.326987458"
Jul 28 19:34:23.720020 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:23.719960    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp" event={"ID":"0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5","Type":"ContainerStarted","Data":"bfde542abf0ca5a830028397ef0df53065f03b0f5c583b79fd77d8bc3b4be80b"}
Jul 28 19:34:23.736841 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:23.736795    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-txhmp" podStartSLOduration=276.259793729 podStartE2EDuration="4m37.736778925s" totalImagesPullingTime="1.476985196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:29:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:34:23.735735688 +0000 UTC m=+289.323398645" watchObservedRunningTime="2026-07-28 19:34:23.736778925 +0000 UTC m=+289.324441870"
Jul 28 19:34:23.834536 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:23.834497    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:23.898871 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:23.898841    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:25.132953 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.132909    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d67fbcb98-bhgfb"]
Jul 28 19:34:25.136577 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.136554    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.148943 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.148917    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d67fbcb98-bhgfb"]
Jul 28 19:34:25.149348 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.149306    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 19:34:25.251675 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.251642    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/db388a9d-815d-415d-a3a2-14aafd18ce4b-console-oauth-config\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.251675 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.251680    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-oauth-serving-cert\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.251893 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.251789    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-console-config\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.251956 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.251892    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-trusted-ca-bundle\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.251956 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.251925    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/db388a9d-815d-415d-a3a2-14aafd18ce4b-console-serving-cert\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.251956 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.251948    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-service-ca\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.252134 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.251978    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffv9q\" (UniqueName: \"kubernetes.io/projected/db388a9d-815d-415d-a3a2-14aafd18ce4b-kube-api-access-ffv9q\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.353019 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.352984    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-trusted-ca-bundle\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.353019 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.353019    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/db388a9d-815d-415d-a3a2-14aafd18ce4b-console-serving-cert\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.353270 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.353040    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-service-ca\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.353270 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.353087    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffv9q\" (UniqueName: \"kubernetes.io/projected/db388a9d-815d-415d-a3a2-14aafd18ce4b-kube-api-access-ffv9q\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.353270 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.353123    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/db388a9d-815d-415d-a3a2-14aafd18ce4b-console-oauth-config\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.353270 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.353151    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-oauth-serving-cert\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.353270 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.353192    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-console-config\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.353826 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.353791    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-service-ca\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.353945 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.353888    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-console-config\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.354030 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.354009    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-trusted-ca-bundle\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.354110 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.354064    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-oauth-serving-cert\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.355931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.355910    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/db388a9d-815d-415d-a3a2-14aafd18ce4b-console-oauth-config\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.356344 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.356319    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/db388a9d-815d-415d-a3a2-14aafd18ce4b-console-serving-cert\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.362865 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.362843    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffv9q\" (UniqueName: \"kubernetes.io/projected/db388a9d-815d-415d-a3a2-14aafd18ce4b-kube-api-access-ffv9q\") pod \"console-d67fbcb98-bhgfb\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") " pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.449939 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.449879    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:25.460740 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.460721    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:25.460856 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.460795    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:34:25.462411 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.462389    2572 patch_prober.go:28] interesting pod/console-69cdff8dd4-d9l29 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.18:8443/health\": dial tcp 10.132.0.18:8443: connect: connection refused" start-of-body=
Jul 28 19:34:25.462503 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.462430    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-69cdff8dd4-d9l29" podUID="cb6decc4-64ba-4211-b929-44e406c4dfca" containerName="console" probeResult="failure" output="Get \"https://10.132.0.18:8443/health\": dial tcp 10.132.0.18:8443: connect: connection refused"
Jul 28 19:34:25.989435 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:25.989407    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d67fbcb98-bhgfb"]
Jul 28 19:34:25.994470 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:34:25.994440    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb388a9d_815d_415d_a3a2_14aafd18ce4b.slice/crio-0873ce18030abba0944c736f2bca25138f7071734abcdc37e186b99bc28b6cad WatchSource:0}: Error finding container 0873ce18030abba0944c736f2bca25138f7071734abcdc37e186b99bc28b6cad: Status 404 returned error can't find the container with id 0873ce18030abba0944c736f2bca25138f7071734abcdc37e186b99bc28b6cad
Jul 28 19:34:26.733140 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:26.733103    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d67fbcb98-bhgfb" event={"ID":"db388a9d-815d-415d-a3a2-14aafd18ce4b","Type":"ContainerStarted","Data":"29cd45b7ba012d5366bce4c6c25afae18ae3184ba187e6d9b80182c1085146e1"}
Jul 28 19:34:26.733607 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:26.733165    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d67fbcb98-bhgfb" event={"ID":"db388a9d-815d-415d-a3a2-14aafd18ce4b","Type":"ContainerStarted","Data":"0873ce18030abba0944c736f2bca25138f7071734abcdc37e186b99bc28b6cad"}
Jul 28 19:34:26.735195 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:26.735167    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7gmsv" event={"ID":"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf","Type":"ContainerStarted","Data":"8b3820453eb1dc85dee9c257ef65c6f3c5ceb7b621318957990b28f166ba5a3d"}
Jul 28 19:34:26.737108 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:26.737087    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6wqlb" event={"ID":"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5","Type":"ContainerStarted","Data":"34d3c270b4aaebca3444cec1f85189397f19fdee8351f172769df2cefc5e39b9"}
Jul 28 19:34:26.737234 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:26.737131    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6wqlb" event={"ID":"eee9f5fc-fb26-4766-86a8-2b2b8b0056c5","Type":"ContainerStarted","Data":"3902f4626ddfe424ec48a4a103c9cb27f234eba85acf1d720dd3aed8af2d2f63"}
Jul 28 19:34:26.737289 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:26.737234    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:34:26.752620 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:26.752569    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d67fbcb98-bhgfb" podStartSLOduration=1.75255028 podStartE2EDuration="1.75255028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:34:26.750823734 +0000 UTC m=+292.338486681" watchObservedRunningTime="2026-07-28 19:34:26.75255028 +0000 UTC m=+292.340213227"
Jul 28 19:34:26.766050 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:26.765995    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6wqlb" podStartSLOduration=253.946495576 podStartE2EDuration="4m17.765976001s" totalImagesPullingTime="3.819480425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:30:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:34:26.76504684 +0000 UTC m=+292.352709786" watchObservedRunningTime="2026-07-28 19:34:26.765976001 +0000 UTC m=+292.353638948"
Jul 28 19:34:29.752397 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:29.752329    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7gmsv" event={"ID":"c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf","Type":"ContainerStarted","Data":"7f289d3162a22b5f637620f5809cb061636888f0ca1f8b756f9c98325f758ca8"}
Jul 28 19:34:29.768628 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:29.768575    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7gmsv" podStartSLOduration=33.257611326 podStartE2EDuration="46.768557629s" totalImagesPullingTime="13.510946303s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:33:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:34:29.768274613 +0000 UTC m=+295.355937559" watchObservedRunningTime="2026-07-28 19:34:29.768557629 +0000 UTC m=+295.356220580"
Jul 28 19:34:29.941151 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:29.941124    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:29.990971 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:29.990948    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:30.858494 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:34:30.858453    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-8b8d6" podUID="303f736b-24a3-4e9b-8904-e4fde37ab7e0"
Jul 28 19:34:33.801998 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:33.801959    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:34.711105 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:34.711061    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 19:34:34.711744 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:34.711723    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 19:34:34.721194 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:34.721174    2572 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 19:34:35.450248 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:35.450220    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:35.452541 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:35.450273    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:35.452541 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:35.451631    2572 patch_prober.go:28] interesting pod/console-d67fbcb98-bhgfb container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.19:8443/health\": dial tcp 10.132.0.19:8443: connect: connection refused" start-of-body=
Jul 28 19:34:35.452541 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:35.451667    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-d67fbcb98-bhgfb" podUID="db388a9d-815d-415d-a3a2-14aafd18ce4b" containerName="console" probeResult="failure" output="Get \"https://10.132.0.19:8443/health\": dial tcp 10.132.0.19:8443: connect: connection refused"
Jul 28 19:34:35.461187 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:35.461162    2572 patch_prober.go:28] interesting pod/console-69cdff8dd4-d9l29 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.18:8443/health\": dial tcp 10.132.0.18:8443: connect: connection refused" start-of-body=
Jul 28 19:34:35.461263 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:35.461202    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-69cdff8dd4-d9l29" podUID="cb6decc4-64ba-4211-b929-44e406c4dfca" containerName="console" probeResult="failure" output="Get \"https://10.132.0.18:8443/health\": dial tcp 10.132.0.18:8443: connect: connection refused"
Jul 28 19:34:35.797284 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:35.797221    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:36.016109 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.016061    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:36.071079 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.071040    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tfqrn"]
Jul 28 19:34:36.074623 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.074597    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.077162 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.077138    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 19:34:36.077893 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.077873    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 19:34:36.077999 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.077947    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ndd7t\""
Jul 28 19:34:36.077999 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.077873    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 19:34:36.078131 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.078056    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 19:34:36.090329 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.090306    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:36.138477 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.138456    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/756f572c-dfa1-4d6d-bffa-7c75f093ec47-sys\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.138477 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.138482    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/756f572c-dfa1-4d6d-bffa-7c75f093ec47-metrics-client-ca\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.138646 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.138508    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/756f572c-dfa1-4d6d-bffa-7c75f093ec47-root\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.138646 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.138564    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/756f572c-dfa1-4d6d-bffa-7c75f093ec47-node-exporter-wtmp\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.138646 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.138603    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwfv5\" (UniqueName: \"kubernetes.io/projected/756f572c-dfa1-4d6d-bffa-7c75f093ec47-kube-api-access-lwfv5\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.138792 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.138640    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/756f572c-dfa1-4d6d-bffa-7c75f093ec47-node-exporter-accelerators-collector-config\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.138792 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.138693    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/756f572c-dfa1-4d6d-bffa-7c75f093ec47-node-exporter-tls\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.138792 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.138734    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/756f572c-dfa1-4d6d-bffa-7c75f093ec47-node-exporter-textfile\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.138792 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.138762    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/756f572c-dfa1-4d6d-bffa-7c75f093ec47-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.146498 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.146475    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:36.239263 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.239229    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/756f572c-dfa1-4d6d-bffa-7c75f093ec47-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.239451 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.239271    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/756f572c-dfa1-4d6d-bffa-7c75f093ec47-sys\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.239451 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.239296    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/756f572c-dfa1-4d6d-bffa-7c75f093ec47-metrics-client-ca\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.239451 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.239327    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/756f572c-dfa1-4d6d-bffa-7c75f093ec47-root\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.239451 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.239347    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/756f572c-dfa1-4d6d-bffa-7c75f093ec47-node-exporter-wtmp\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.239451 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.239368    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/756f572c-dfa1-4d6d-bffa-7c75f093ec47-sys\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.239451 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.239375    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwfv5\" (UniqueName: \"kubernetes.io/projected/756f572c-dfa1-4d6d-bffa-7c75f093ec47-kube-api-access-lwfv5\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.239766 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.239443    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/756f572c-dfa1-4d6d-bffa-7c75f093ec47-root\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.239766 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.239455    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/756f572c-dfa1-4d6d-bffa-7c75f093ec47-node-exporter-accelerators-collector-config\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.239766 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.239524    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/756f572c-dfa1-4d6d-bffa-7c75f093ec47-node-exporter-wtmp\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.239766 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.239531    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/756f572c-dfa1-4d6d-bffa-7c75f093ec47-node-exporter-tls\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.239766 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.239596    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/756f572c-dfa1-4d6d-bffa-7c75f093ec47-node-exporter-textfile\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.240291 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.239998    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/756f572c-dfa1-4d6d-bffa-7c75f093ec47-node-exporter-accelerators-collector-config\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.240291 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.240238    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/756f572c-dfa1-4d6d-bffa-7c75f093ec47-node-exporter-textfile\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.240482 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.240461    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/756f572c-dfa1-4d6d-bffa-7c75f093ec47-metrics-client-ca\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.242209 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.242177    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/756f572c-dfa1-4d6d-bffa-7c75f093ec47-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.242322 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.242290    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/756f572c-dfa1-4d6d-bffa-7c75f093ec47-node-exporter-tls\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.247780 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.247755    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:36.248976 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.248935    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwfv5\" (UniqueName: \"kubernetes.io/projected/756f572c-dfa1-4d6d-bffa-7c75f093ec47-kube-api-access-lwfv5\") pod \"node-exporter-tfqrn\" (UID: \"756f572c-dfa1-4d6d-bffa-7c75f093ec47\") " pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.347444 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.347341    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:36.385382 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.385351    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tfqrn"
Jul 28 19:34:36.399002 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:34:36.398937    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod756f572c_dfa1_4d6d_bffa_7c75f093ec47.slice/crio-8681a810741724ad9acb56c94dc7180af83ae6f426c091733d89637c2fdd6996 WatchSource:0}: Error finding container 8681a810741724ad9acb56c94dc7180af83ae6f426c091733d89637c2fdd6996: Status 404 returned error can't find the container with id 8681a810741724ad9acb56c94dc7180af83ae6f426c091733d89637c2fdd6996
Jul 28 19:34:36.401186 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.401167    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:34:36.743543 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.743453    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6wqlb"
Jul 28 19:34:36.779909 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:36.779867    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tfqrn" event={"ID":"756f572c-dfa1-4d6d-bffa-7c75f093ec47","Type":"ContainerStarted","Data":"8681a810741724ad9acb56c94dc7180af83ae6f426c091733d89637c2fdd6996"}
Jul 28 19:34:37.540284 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:37.540256    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:37.599841 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:37.599789    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:37.785250 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:37.785224    2572 generic.go:356] "Generic (PLEG): container finished" podID="756f572c-dfa1-4d6d-bffa-7c75f093ec47" containerID="6e45437bd4fe68b883e0793560425ec78f62519285f763b7afd0c82d4d30ea79" exitCode=0
Jul 28 19:34:37.785543 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:37.785304    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tfqrn" event={"ID":"756f572c-dfa1-4d6d-bffa-7c75f093ec47","Type":"ContainerDied","Data":"6e45437bd4fe68b883e0793560425ec78f62519285f763b7afd0c82d4d30ea79"}
Jul 28 19:34:38.791261 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:38.791226    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tfqrn" event={"ID":"756f572c-dfa1-4d6d-bffa-7c75f093ec47","Type":"ContainerStarted","Data":"8ce5532f7fbba0639ab3b3da44488ca81a1645c3e0b3188db5cac75bd3f473ac"}
Jul 28 19:34:38.791636 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:38.791288    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tfqrn" event={"ID":"756f572c-dfa1-4d6d-bffa-7c75f093ec47","Type":"ContainerStarted","Data":"f863e0e98718b5c647750a6c0fd111b0016dd055610e9c6901d1996f102c5d92"}
Jul 28 19:34:38.811803 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:38.811753    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tfqrn" podStartSLOduration=1.820349963 podStartE2EDuration="2.811739281s" totalImagesPullingTime="929.390267ms" totalInitContainerRuntime="61.999051ms" isStatefulPod=true podCreationTimestamp="2026-07-28 19:34:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:34:38.809587753 +0000 UTC m=+304.397250722" watchObservedRunningTime="2026-07-28 19:34:38.811739281 +0000 UTC m=+304.399402227"
Jul 28 19:34:38.992764 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:38.992738    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:39.082164 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:39.082137    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:39.346653 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:39.346583    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:39.406420 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:39.406379    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:39.432031 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:39.432009    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9fc54f48b-9jtb2"]
Jul 28 19:34:39.436494 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:39.436469    2572 patch_prober.go:28] interesting pod/image-registry-9fc54f48b-9jtb2 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 28 19:34:39.436619 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:39.436513    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2" podUID="f427e901-fd2a-4d35-a853-fe83fabbca35" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 19:34:39.527163 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:39.527133    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:39.586252 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:39.586226    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:39.791613 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:39.791556    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:40.837754 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.837722    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-q26gl"]
Jul 28 19:34:40.841880 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.841853    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-q26gl"
Jul 28 19:34:40.843405 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.843380    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69cdff8dd4-d9l29"]
Jul 28 19:34:40.844592 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.844569    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 28 19:34:40.844692 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.844628    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-8rs6l\""
Jul 28 19:34:40.847526 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.847504    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:40.851892 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.851852    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-q26gl"]
Jul 28 19:34:40.887993 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.887967    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55f57c4956-sxppv"]
Jul 28 19:34:40.890972 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.890957    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:40.902541 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.902521    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55f57c4956-sxppv"]
Jul 28 19:34:40.976861 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.976833    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prz9c\" (UniqueName: \"kubernetes.io/projected/18b6ab88-4bb2-4798-b7f4-5755b6802b53-kube-api-access-prz9c\") pod \"monitoring-plugin-68b7fc9574-q26gl\" (UID: \"18b6ab88-4bb2-4798-b7f4-5755b6802b53\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-q26gl"
Jul 28 19:34:40.977012 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.976881    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-oauth-serving-cert\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:40.977012 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.976944    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/18b6ab88-4bb2-4798-b7f4-5755b6802b53-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-q26gl\" (UID: \"18b6ab88-4bb2-4798-b7f4-5755b6802b53\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-q26gl"
Jul 28 19:34:40.977125 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.977014    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-console-serving-cert\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:40.977125 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.977031    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-service-ca\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:40.977125 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.977048    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mkqb\" (UniqueName: \"kubernetes.io/projected/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-kube-api-access-2mkqb\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:40.977125 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.977114    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-console-oauth-config\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:40.977250 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.977147    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-console-config\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:40.977250 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:40.977198    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-trusted-ca-bundle\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:41.040267 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.040239    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:41.078196 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.078170    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/18b6ab88-4bb2-4798-b7f4-5755b6802b53-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-q26gl\" (UID: \"18b6ab88-4bb2-4798-b7f4-5755b6802b53\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-q26gl"
Jul 28 19:34:41.078324 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.078231    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-console-serving-cert\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:41.078324 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.078251    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-service-ca\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:41.078324 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.078267    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mkqb\" (UniqueName: \"kubernetes.io/projected/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-kube-api-access-2mkqb\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:41.078324 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.078288    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-console-oauth-config\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:41.078324 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.078311    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-console-config\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:41.078324 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:34:41.078314    2572 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 28 19:34:41.078627 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.078345    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-trusted-ca-bundle\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:41.078627 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:34:41.078374    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18b6ab88-4bb2-4798-b7f4-5755b6802b53-monitoring-plugin-cert podName:18b6ab88-4bb2-4798-b7f4-5755b6802b53 nodeName:}" failed. No retries permitted until 2026-07-28 19:34:41.578354137 +0000 UTC m=+307.166017061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/18b6ab88-4bb2-4798-b7f4-5755b6802b53-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-q26gl" (UID: "18b6ab88-4bb2-4798-b7f4-5755b6802b53") : secret "monitoring-plugin-cert" not found
Jul 28 19:34:41.078627 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.078425    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prz9c\" (UniqueName: \"kubernetes.io/projected/18b6ab88-4bb2-4798-b7f4-5755b6802b53-kube-api-access-prz9c\") pod \"monitoring-plugin-68b7fc9574-q26gl\" (UID: \"18b6ab88-4bb2-4798-b7f4-5755b6802b53\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-q26gl"
Jul 28 19:34:41.078627 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.078471    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-oauth-serving-cert\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:41.079170 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.079140    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-console-config\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:41.079327 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.079306    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-oauth-serving-cert\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:41.079396 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.079310    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-service-ca\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:41.079396 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.079350    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-trusted-ca-bundle\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:41.080826 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.080801    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-console-serving-cert\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:41.081404 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.081389    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-console-oauth-config\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:41.088685 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.088632    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mkqb\" (UniqueName: \"kubernetes.io/projected/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-kube-api-access-2mkqb\") pod \"console-55f57c4956-sxppv\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") " pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:41.089059 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.089039    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prz9c\" (UniqueName: \"kubernetes.io/projected/18b6ab88-4bb2-4798-b7f4-5755b6802b53-kube-api-access-prz9c\") pod \"monitoring-plugin-68b7fc9574-q26gl\" (UID: \"18b6ab88-4bb2-4798-b7f4-5755b6802b53\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-q26gl"
Jul 28 19:34:41.140475 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.140438    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:41.199263 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.199225    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:41.238853 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.238825    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:41.242368 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.242343    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"]
Jul 28 19:34:41.245818 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.245796    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.248950 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.248855    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 28 19:34:41.248950 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.248886    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 28 19:34:41.249157 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.249145    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 28 19:34:41.249211 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.249198    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 28 19:34:41.249457 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.249336    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 28 19:34:41.249601 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.249530    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-l9mbg\""
Jul 28 19:34:41.254038 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.254011    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 28 19:34:41.261105 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.261061    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"]
Jul 28 19:34:41.329277 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.329189    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55f57c4956-sxppv"]
Jul 28 19:34:41.331718 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:34:41.331695    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e3f3d7_8b49_4df7_bde0_d8ed7252c9d7.slice/crio-05c17c3492060771a3226b8a2f173a5c94e1c29e46d16c0ef5a547a844e459ec WatchSource:0}: Error finding container 05c17c3492060771a3226b8a2f173a5c94e1c29e46d16c0ef5a547a844e459ec: Status 404 returned error can't find the container with id 05c17c3492060771a3226b8a2f173a5c94e1c29e46d16c0ef5a547a844e459ec
Jul 28 19:34:41.382046 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.382027    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/28abda71-7951-4756-8c6a-13cdba055a55-secret-telemeter-client\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.382163 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.382052    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j8tb\" (UniqueName: \"kubernetes.io/projected/28abda71-7951-4756-8c6a-13cdba055a55-kube-api-access-9j8tb\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.382163 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.382120    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/28abda71-7951-4756-8c6a-13cdba055a55-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.382163 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.382148    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28abda71-7951-4756-8c6a-13cdba055a55-metrics-client-ca\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.382325 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.382226    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/28abda71-7951-4756-8c6a-13cdba055a55-federate-client-tls\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.382325 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.382257    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28abda71-7951-4756-8c6a-13cdba055a55-telemeter-trusted-ca-bundle\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.382325 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.382306    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/28abda71-7951-4756-8c6a-13cdba055a55-telemeter-client-tls\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.382473 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.382329    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28abda71-7951-4756-8c6a-13cdba055a55-serving-certs-ca-bundle\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.482927 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.482897    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/28abda71-7951-4756-8c6a-13cdba055a55-telemeter-client-tls\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.483056 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.482933    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28abda71-7951-4756-8c6a-13cdba055a55-serving-certs-ca-bundle\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.483056 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.482968    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/28abda71-7951-4756-8c6a-13cdba055a55-secret-telemeter-client\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.483056 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.482994    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j8tb\" (UniqueName: \"kubernetes.io/projected/28abda71-7951-4756-8c6a-13cdba055a55-kube-api-access-9j8tb\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.483056 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.483041    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/28abda71-7951-4756-8c6a-13cdba055a55-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.483275 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.483100    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28abda71-7951-4756-8c6a-13cdba055a55-metrics-client-ca\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.483275 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.483247    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/28abda71-7951-4756-8c6a-13cdba055a55-federate-client-tls\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.483373 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.483284    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28abda71-7951-4756-8c6a-13cdba055a55-telemeter-trusted-ca-bundle\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.483834 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.483807    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28abda71-7951-4756-8c6a-13cdba055a55-metrics-client-ca\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.483970 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.483814    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28abda71-7951-4756-8c6a-13cdba055a55-serving-certs-ca-bundle\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.484180 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.484154    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28abda71-7951-4756-8c6a-13cdba055a55-telemeter-trusted-ca-bundle\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.485533 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.485509    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/28abda71-7951-4756-8c6a-13cdba055a55-telemeter-client-tls\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.485646 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.485629    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/28abda71-7951-4756-8c6a-13cdba055a55-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.485716 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.485696    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/28abda71-7951-4756-8c6a-13cdba055a55-secret-telemeter-client\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.485752 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.485698    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/28abda71-7951-4756-8c6a-13cdba055a55-federate-client-tls\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.491077 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.491050    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j8tb\" (UniqueName: \"kubernetes.io/projected/28abda71-7951-4756-8c6a-13cdba055a55-kube-api-access-9j8tb\") pod \"telemeter-client-66fcc6df55-n9zh2\" (UID: \"28abda71-7951-4756-8c6a-13cdba055a55\") " pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.558566 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.558544    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"
Jul 28 19:34:41.584101 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.584078    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/18b6ab88-4bb2-4798-b7f4-5755b6802b53-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-q26gl\" (UID: \"18b6ab88-4bb2-4798-b7f4-5755b6802b53\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-q26gl"
Jul 28 19:34:41.586414 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.586398    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/18b6ab88-4bb2-4798-b7f4-5755b6802b53-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-q26gl\" (UID: \"18b6ab88-4bb2-4798-b7f4-5755b6802b53\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-q26gl"
Jul 28 19:34:41.687686 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.687606    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-66fcc6df55-n9zh2"]
Jul 28 19:34:41.690493 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:34:41.690469    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28abda71_7951_4756_8c6a_13cdba055a55.slice/crio-92dfb25e2ed12af9230a75d98ba8fae77d02ededd4399e065ac3a816d34c676b WatchSource:0}: Error finding container 92dfb25e2ed12af9230a75d98ba8fae77d02ededd4399e065ac3a816d34c676b: Status 404 returned error can't find the container with id 92dfb25e2ed12af9230a75d98ba8fae77d02ededd4399e065ac3a816d34c676b
Jul 28 19:34:41.753604 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.753581    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-q26gl"
Jul 28 19:34:41.804561 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.804538    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55f57c4956-sxppv" event={"ID":"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7","Type":"ContainerStarted","Data":"e7de0eab59c24422fad579b96eec607c6def225f734b3254670ccf490ffd93a4"}
Jul 28 19:34:41.804661 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.804590    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55f57c4956-sxppv" event={"ID":"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7","Type":"ContainerStarted","Data":"05c17c3492060771a3226b8a2f173a5c94e1c29e46d16c0ef5a547a844e459ec"}
Jul 28 19:34:41.807389 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.807361    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2" event={"ID":"28abda71-7951-4756-8c6a-13cdba055a55","Type":"ContainerStarted","Data":"92dfb25e2ed12af9230a75d98ba8fae77d02ededd4399e065ac3a816d34c676b"}
Jul 28 19:34:41.822779 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.822739    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55f57c4956-sxppv" podStartSLOduration=1.822723489 podStartE2EDuration="1.822723489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:34:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:34:41.821970987 +0000 UTC m=+307.409633934" watchObservedRunningTime="2026-07-28 19:34:41.822723489 +0000 UTC m=+307.410386435"
Jul 28 19:34:41.876176 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:41.876121    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-q26gl"]
Jul 28 19:34:41.879194 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:34:41.879171    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18b6ab88_4bb2_4798_b7f4_5755b6802b53.slice/crio-e0cd33d94242b1311d7d7da4450e2dc4e19320ed7d3c1124cc14a207ad0bbcc9 WatchSource:0}: Error finding container e0cd33d94242b1311d7d7da4450e2dc4e19320ed7d3c1124cc14a207ad0bbcc9: Status 404 returned error can't find the container with id e0cd33d94242b1311d7d7da4450e2dc4e19320ed7d3c1124cc14a207ad0bbcc9
Jul 28 19:34:42.304230 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:42.304150    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:42.813791 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:42.813754    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-q26gl" event={"ID":"18b6ab88-4bb2-4798-b7f4-5755b6802b53","Type":"ContainerStarted","Data":"e0cd33d94242b1311d7d7da4450e2dc4e19320ed7d3c1124cc14a207ad0bbcc9"}
Jul 28 19:34:43.153344 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:43.153269    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:43.206545 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:43.206510    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:43.858598 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:43.858562    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:34:44.825756 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:44.825662    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-q26gl" event={"ID":"18b6ab88-4bb2-4798-b7f4-5755b6802b53","Type":"ContainerStarted","Data":"b5ff0aa3f797cb70ccb09190744b5e9f530d21ca2bb44bad8138881475524a31"}
Jul 28 19:34:44.825756 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:44.825720    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-q26gl"
Jul 28 19:34:44.828272 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:44.828244    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2" event={"ID":"28abda71-7951-4756-8c6a-13cdba055a55","Type":"ContainerStarted","Data":"03a724d27bee502e1c0907f2aa99ee2fea087c1a916009a0e249cdd355381a04"}
Jul 28 19:34:44.832546 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:44.832520    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-q26gl"
Jul 28 19:34:44.850009 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:44.849963    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-q26gl" podStartSLOduration=2.799902381 podStartE2EDuration="4.849949193s" totalImagesPullingTime="2.050046812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:34:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:34:44.848760683 +0000 UTC m=+310.436423628" watchObservedRunningTime="2026-07-28 19:34:44.849949193 +0000 UTC m=+310.437612190"
Jul 28 19:34:45.454975 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:45.454903    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:45.458870 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:45.458845    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:34:45.615813 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:45.615782    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:45.677455 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:45.677422    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:45.787800 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:45.787722    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:45.833862 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:45.833828    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2" event={"ID":"28abda71-7951-4756-8c6a-13cdba055a55","Type":"ContainerStarted","Data":"d05d1bca3e763f13c8784a8c2dc30e7e432cdf4b6e8e40463292361f64b03b97"}
Jul 28 19:34:45.834233 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:45.833883    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2" event={"ID":"28abda71-7951-4756-8c6a-13cdba055a55","Type":"ContainerStarted","Data":"2ae972ad6b158209ee628ccb4bc3fa212be1e9aeca86e2aac53265704e7b10ca"}
Jul 28 19:34:45.856666 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:45.856617    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-66fcc6df55-n9zh2" podStartSLOduration=1.5298710789999999 podStartE2EDuration="4.856596195s" totalImagesPullingTime="3.326725116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:34:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:34:45.855243807 +0000 UTC m=+311.442906755" watchObservedRunningTime="2026-07-28 19:34:45.856596195 +0000 UTC m=+311.444259143"
Jul 28 19:34:45.859567 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:45.859545    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:46.882279 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:46.882252    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:46.912954 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:46.912931    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55f57c4956-sxppv"]
Jul 28 19:34:46.945736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:46.945712    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5556f97b8b-dnc78"]
Jul 28 19:34:46.949925 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:46.949909    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:46.964408 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:46.964389    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5556f97b8b-dnc78"]
Jul 28 19:34:46.985140 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:46.985120    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:47.132723 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.132665    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-trusted-ca-bundle\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.132723 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.132693    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-oauth-serving-cert\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.132883 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.132734    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aed855e7-36d0-4bbc-b233-7c93dcb687dd-console-oauth-config\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.132883 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.132790    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aed855e7-36d0-4bbc-b233-7c93dcb687dd-console-serving-cert\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.132883 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.132819    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-service-ca\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.133036 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.132924    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-console-config\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.133036 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.132970    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hczlx\" (UniqueName: \"kubernetes.io/projected/aed855e7-36d0-4bbc-b233-7c93dcb687dd-kube-api-access-hczlx\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.233301 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.233276    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aed855e7-36d0-4bbc-b233-7c93dcb687dd-console-serving-cert\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.233414 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.233307    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-service-ca\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.233414 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.233346    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-console-config\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.233414 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.233376    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hczlx\" (UniqueName: \"kubernetes.io/projected/aed855e7-36d0-4bbc-b233-7c93dcb687dd-kube-api-access-hczlx\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.233414 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.233407    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-trusted-ca-bundle\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.233616 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.233430    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-oauth-serving-cert\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.233616 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.233499    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aed855e7-36d0-4bbc-b233-7c93dcb687dd-console-oauth-config\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.234185 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.234161    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-console-config\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.234265 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.234161    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-oauth-serving-cert\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.234639 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.234615    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-service-ca\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.234763 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.234744    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-trusted-ca-bundle\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.235717 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.235690    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aed855e7-36d0-4bbc-b233-7c93dcb687dd-console-serving-cert\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.235796 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.235758    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aed855e7-36d0-4bbc-b233-7c93dcb687dd-console-oauth-config\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.241184 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.241163    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hczlx\" (UniqueName: \"kubernetes.io/projected/aed855e7-36d0-4bbc-b233-7c93dcb687dd-kube-api-access-hczlx\") pod \"console-5556f97b8b-dnc78\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") " pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.258983 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.258961    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:47.408914 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.408893    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5556f97b8b-dnc78"]
Jul 28 19:34:47.410847 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:34:47.410822    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaed855e7_36d0_4bbc_b233_7c93dcb687dd.slice/crio-b0e1d10f48825d0b7fffecaddf360ea162ca83e8793226ee3a593dfdafd98535 WatchSource:0}: Error finding container b0e1d10f48825d0b7fffecaddf360ea162ca83e8793226ee3a593dfdafd98535: Status 404 returned error can't find the container with id b0e1d10f48825d0b7fffecaddf360ea162ca83e8793226ee3a593dfdafd98535
Jul 28 19:34:47.842736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.842700    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5556f97b8b-dnc78" event={"ID":"aed855e7-36d0-4bbc-b233-7c93dcb687dd","Type":"ContainerStarted","Data":"4ba664f0d928ec5a938d96630fee513cba99a71e72fb9442f84d47bd6a80a47d"}
Jul 28 19:34:47.842926 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.842752    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5556f97b8b-dnc78" event={"ID":"aed855e7-36d0-4bbc-b233-7c93dcb687dd","Type":"ContainerStarted","Data":"b0e1d10f48825d0b7fffecaddf360ea162ca83e8793226ee3a593dfdafd98535"}
Jul 28 19:34:47.860740 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:47.860681    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5556f97b8b-dnc78" podStartSLOduration=1.860664285 podStartE2EDuration="1.860664285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:34:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:34:47.858940364 +0000 UTC m=+313.446603311" watchObservedRunningTime="2026-07-28 19:34:47.860664285 +0000 UTC m=+313.448327231"
Jul 28 19:34:49.436340 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:49.436315    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:34:50.140130 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:50.140057    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:50.240253 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:50.240231    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:51.199510 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:51.199480    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:34:51.356147 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:51.356118    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:51.415057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:51.415031    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:53.867220 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:53.867190    2572 generic.go:356] "Generic (PLEG): container finished" podID="ba18d3b8-a8fb-4d09-bc33-c5213185a953" containerID="76e77dd6204e56746386cc36f1d176073c93a0d7541b75aac349f74ad72db11c" exitCode=0
Jul 28 19:34:53.867576 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:53.867258    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-q8b99" event={"ID":"ba18d3b8-a8fb-4d09-bc33-c5213185a953","Type":"ContainerDied","Data":"76e77dd6204e56746386cc36f1d176073c93a0d7541b75aac349f74ad72db11c"}
Jul 28 19:34:53.867617 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:53.867589    2572 scope.go:117] "RemoveContainer" containerID="76e77dd6204e56746386cc36f1d176073c93a0d7541b75aac349f74ad72db11c"
Jul 28 19:34:54.794652 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:54.794618    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6wqlb_eee9f5fc-fb26-4766-86a8-2b2b8b0056c5/dns/0.log"
Jul 28 19:34:54.873011 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:54.872981    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-q8b99" event={"ID":"ba18d3b8-a8fb-4d09-bc33-c5213185a953","Type":"ContainerStarted","Data":"e4e1d7ee5e92e2fddb8293618445dc60248ee77c2257329d7b3159bd0a0fac6b"}
Jul 28 19:34:54.874215 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:54.874193    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:54.974722 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:54.974700    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:54.995459 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:54.995424    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6wqlb_eee9f5fc-fb26-4766-86a8-2b2b8b0056c5/kube-rbac-proxy/0.log"
Jul 28 19:34:56.397542 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:56.397520    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-td8fx_4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5/dns-node-resolver/0.log"
Jul 28 19:34:57.193840 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:57.193786    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tpngt_a5c7f30b-8bac-4127-9e0a-44c45b92e47b/serve-healthcheck-canary/0.log"
Jul 28 19:34:57.259459 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:57.259427    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:57.259597 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:57.259486    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:57.263811 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:57.263793    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:57.887469 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:57.887438    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:34:57.932170 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:57.932143    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d67fbcb98-bhgfb"]
Jul 28 19:34:58.035658 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:58.035626    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:58.136926 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:58.136903    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:58.237393 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:34:58.237331    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:04.450642 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.450601    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2" podUID="f427e901-fd2a-4d35-a853-fe83fabbca35" containerName="registry" containerID="cri-o://d945c6fba34b66e465cacc9d117cf1d7f29d07cf21fc38bb46674401d239acd6" gracePeriod=30
Jul 28 19:35:04.693424 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.693404    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:35:04.762291 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.762239    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-certificates\") pod \"f427e901-fd2a-4d35-a853-fe83fabbca35\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") "
Jul 28 19:35:04.762402 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.762292    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls\") pod \"f427e901-fd2a-4d35-a853-fe83fabbca35\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") "
Jul 28 19:35:04.762402 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.762318    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f427e901-fd2a-4d35-a853-fe83fabbca35-image-registry-private-configuration\") pod \"f427e901-fd2a-4d35-a853-fe83fabbca35\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") "
Jul 28 19:35:04.762402 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.762338    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f427e901-fd2a-4d35-a853-fe83fabbca35-installation-pull-secrets\") pod \"f427e901-fd2a-4d35-a853-fe83fabbca35\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") "
Jul 28 19:35:04.762402 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.762361    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f427e901-fd2a-4d35-a853-fe83fabbca35-trusted-ca\") pod \"f427e901-fd2a-4d35-a853-fe83fabbca35\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") "
Jul 28 19:35:04.762402 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.762376    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5x7t\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-kube-api-access-x5x7t\") pod \"f427e901-fd2a-4d35-a853-fe83fabbca35\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") "
Jul 28 19:35:04.762667 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.762426    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f427e901-fd2a-4d35-a853-fe83fabbca35-ca-trust-extracted\") pod \"f427e901-fd2a-4d35-a853-fe83fabbca35\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") "
Jul 28 19:35:04.762667 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.762459    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-bound-sa-token\") pod \"f427e901-fd2a-4d35-a853-fe83fabbca35\" (UID: \"f427e901-fd2a-4d35-a853-fe83fabbca35\") "
Jul 28 19:35:04.762761 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.762695    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f427e901-fd2a-4d35-a853-fe83fabbca35" (UID: "f427e901-fd2a-4d35-a853-fe83fabbca35"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:35:04.762983 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.762854    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f427e901-fd2a-4d35-a853-fe83fabbca35-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f427e901-fd2a-4d35-a853-fe83fabbca35" (UID: "f427e901-fd2a-4d35-a853-fe83fabbca35"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:35:04.764480 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.764460    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f427e901-fd2a-4d35-a853-fe83fabbca35" (UID: "f427e901-fd2a-4d35-a853-fe83fabbca35"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:35:04.764867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.764813    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f427e901-fd2a-4d35-a853-fe83fabbca35-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f427e901-fd2a-4d35-a853-fe83fabbca35" (UID: "f427e901-fd2a-4d35-a853-fe83fabbca35"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:35:04.765054 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.765030    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f427e901-fd2a-4d35-a853-fe83fabbca35" (UID: "f427e901-fd2a-4d35-a853-fe83fabbca35"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:35:04.765200 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.765174    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-kube-api-access-x5x7t" (OuterVolumeSpecName: "kube-api-access-x5x7t") pod "f427e901-fd2a-4d35-a853-fe83fabbca35" (UID: "f427e901-fd2a-4d35-a853-fe83fabbca35"). InnerVolumeSpecName "kube-api-access-x5x7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:35:04.765286 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.765182    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f427e901-fd2a-4d35-a853-fe83fabbca35-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f427e901-fd2a-4d35-a853-fe83fabbca35" (UID: "f427e901-fd2a-4d35-a853-fe83fabbca35"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:35:04.772386 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.772362    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f427e901-fd2a-4d35-a853-fe83fabbca35-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f427e901-fd2a-4d35-a853-fe83fabbca35" (UID: "f427e901-fd2a-4d35-a853-fe83fabbca35"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:35:04.863428 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.863401    2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-tls\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:04.863428 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.863422    2572 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f427e901-fd2a-4d35-a853-fe83fabbca35-image-registry-private-configuration\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:04.863428 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.863432    2572 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f427e901-fd2a-4d35-a853-fe83fabbca35-installation-pull-secrets\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:04.863761 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.863440    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f427e901-fd2a-4d35-a853-fe83fabbca35-trusted-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:04.863761 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.863455    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5x7t\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-kube-api-access-x5x7t\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:04.863761 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.863463    2572 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f427e901-fd2a-4d35-a853-fe83fabbca35-ca-trust-extracted\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:04.863761 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.863471    2572 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f427e901-fd2a-4d35-a853-fe83fabbca35-bound-sa-token\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:04.863761 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.863480    2572 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f427e901-fd2a-4d35-a853-fe83fabbca35-registry-certificates\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:04.910180 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.910150    2572 generic.go:356] "Generic (PLEG): container finished" podID="f427e901-fd2a-4d35-a853-fe83fabbca35" containerID="d945c6fba34b66e465cacc9d117cf1d7f29d07cf21fc38bb46674401d239acd6" exitCode=0
Jul 28 19:35:04.910326 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.910209    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2"
Jul 28 19:35:04.910326 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.910246    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2" event={"ID":"f427e901-fd2a-4d35-a853-fe83fabbca35","Type":"ContainerDied","Data":"d945c6fba34b66e465cacc9d117cf1d7f29d07cf21fc38bb46674401d239acd6"}
Jul 28 19:35:04.910423 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.910347    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9fc54f48b-9jtb2" event={"ID":"f427e901-fd2a-4d35-a853-fe83fabbca35","Type":"ContainerDied","Data":"29e873e68f91f6f8196d9c0c90714f19e9bd69a288524dce955e3be25dfecf67"}
Jul 28 19:35:04.910423 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.910380    2572 scope.go:117] "RemoveContainer" containerID="d945c6fba34b66e465cacc9d117cf1d7f29d07cf21fc38bb46674401d239acd6"
Jul 28 19:35:04.918422 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.918404    2572 scope.go:117] "RemoveContainer" containerID="d945c6fba34b66e465cacc9d117cf1d7f29d07cf21fc38bb46674401d239acd6"
Jul 28 19:35:04.918692 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:35:04.918662    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d945c6fba34b66e465cacc9d117cf1d7f29d07cf21fc38bb46674401d239acd6\": container with ID starting with d945c6fba34b66e465cacc9d117cf1d7f29d07cf21fc38bb46674401d239acd6 not found: ID does not exist" containerID="d945c6fba34b66e465cacc9d117cf1d7f29d07cf21fc38bb46674401d239acd6"
Jul 28 19:35:04.918777 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.918692    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d945c6fba34b66e465cacc9d117cf1d7f29d07cf21fc38bb46674401d239acd6"} err="failed to get container status \"d945c6fba34b66e465cacc9d117cf1d7f29d07cf21fc38bb46674401d239acd6\": rpc error: code = NotFound desc = could not find container \"d945c6fba34b66e465cacc9d117cf1d7f29d07cf21fc38bb46674401d239acd6\": container with ID starting with d945c6fba34b66e465cacc9d117cf1d7f29d07cf21fc38bb46674401d239acd6 not found: ID does not exist"
Jul 28 19:35:04.927868 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.927830    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9fc54f48b-9jtb2"]
Jul 28 19:35:04.930641 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:04.930623    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-9fc54f48b-9jtb2"]
Jul 28 19:35:05.865563 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:05.865519    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-69cdff8dd4-d9l29" podUID="cb6decc4-64ba-4211-b929-44e406c4dfca" containerName="console" containerID="cri-o://cdd3a97e446f1ef25a264a34269b47f44b592af059031d974673d56e21ae8fec" gracePeriod=15
Jul 28 19:35:06.126325 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.126290    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:35:06.178241 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.178214    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb6decc4-64ba-4211-b929-44e406c4dfca-console-config\") pod \"cb6decc4-64ba-4211-b929-44e406c4dfca\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") "
Jul 28 19:35:06.178415 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.178261    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb6decc4-64ba-4211-b929-44e406c4dfca-console-serving-cert\") pod \"cb6decc4-64ba-4211-b929-44e406c4dfca\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") "
Jul 28 19:35:06.178415 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.178290    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb6decc4-64ba-4211-b929-44e406c4dfca-console-oauth-config\") pod \"cb6decc4-64ba-4211-b929-44e406c4dfca\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") "
Jul 28 19:35:06.178415 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.178385    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb6decc4-64ba-4211-b929-44e406c4dfca-service-ca\") pod \"cb6decc4-64ba-4211-b929-44e406c4dfca\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") "
Jul 28 19:35:06.178577 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.178416    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb6decc4-64ba-4211-b929-44e406c4dfca-oauth-serving-cert\") pod \"cb6decc4-64ba-4211-b929-44e406c4dfca\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") "
Jul 28 19:35:06.178577 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.178501    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7vgw\" (UniqueName: \"kubernetes.io/projected/cb6decc4-64ba-4211-b929-44e406c4dfca-kube-api-access-t7vgw\") pod \"cb6decc4-64ba-4211-b929-44e406c4dfca\" (UID: \"cb6decc4-64ba-4211-b929-44e406c4dfca\") "
Jul 28 19:35:06.178676 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.178602    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb6decc4-64ba-4211-b929-44e406c4dfca-console-config" (OuterVolumeSpecName: "console-config") pod "cb6decc4-64ba-4211-b929-44e406c4dfca" (UID: "cb6decc4-64ba-4211-b929-44e406c4dfca"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:35:06.178731 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.178698    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb6decc4-64ba-4211-b929-44e406c4dfca-service-ca" (OuterVolumeSpecName: "service-ca") pod "cb6decc4-64ba-4211-b929-44e406c4dfca" (UID: "cb6decc4-64ba-4211-b929-44e406c4dfca"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:35:06.178808 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.178787    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb6decc4-64ba-4211-b929-44e406c4dfca-console-config\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:06.178865 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.178812    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb6decc4-64ba-4211-b929-44e406c4dfca-service-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:06.178865 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.178815    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb6decc4-64ba-4211-b929-44e406c4dfca-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cb6decc4-64ba-4211-b929-44e406c4dfca" (UID: "cb6decc4-64ba-4211-b929-44e406c4dfca"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:35:06.180677 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.180653    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb6decc4-64ba-4211-b929-44e406c4dfca-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cb6decc4-64ba-4211-b929-44e406c4dfca" (UID: "cb6decc4-64ba-4211-b929-44e406c4dfca"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:35:06.180915 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.180897    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb6decc4-64ba-4211-b929-44e406c4dfca-kube-api-access-t7vgw" (OuterVolumeSpecName: "kube-api-access-t7vgw") pod "cb6decc4-64ba-4211-b929-44e406c4dfca" (UID: "cb6decc4-64ba-4211-b929-44e406c4dfca"). InnerVolumeSpecName "kube-api-access-t7vgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:35:06.181153 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.181135    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb6decc4-64ba-4211-b929-44e406c4dfca-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cb6decc4-64ba-4211-b929-44e406c4dfca" (UID: "cb6decc4-64ba-4211-b929-44e406c4dfca"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:35:06.279972 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.279941    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb6decc4-64ba-4211-b929-44e406c4dfca-console-oauth-config\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:06.279972 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.279974    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb6decc4-64ba-4211-b929-44e406c4dfca-oauth-serving-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:06.280164 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.279989    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7vgw\" (UniqueName: \"kubernetes.io/projected/cb6decc4-64ba-4211-b929-44e406c4dfca-kube-api-access-t7vgw\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:06.280164 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.280003    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb6decc4-64ba-4211-b929-44e406c4dfca-console-serving-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:06.349847 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.349813    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:06.449455 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.449386    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:06.863755 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.863724    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f427e901-fd2a-4d35-a853-fe83fabbca35" path="/var/lib/kubelet/pods/f427e901-fd2a-4d35-a853-fe83fabbca35/volumes"
Jul 28 19:35:06.926991 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.926964    2572 generic.go:356] "Generic (PLEG): container finished" podID="cb6decc4-64ba-4211-b929-44e406c4dfca" containerID="cdd3a97e446f1ef25a264a34269b47f44b592af059031d974673d56e21ae8fec" exitCode=0
Jul 28 19:35:06.927416 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.927001    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69cdff8dd4-d9l29"
Jul 28 19:35:06.927416 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.927007    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69cdff8dd4-d9l29" event={"ID":"cb6decc4-64ba-4211-b929-44e406c4dfca","Type":"ContainerDied","Data":"cdd3a97e446f1ef25a264a34269b47f44b592af059031d974673d56e21ae8fec"}
Jul 28 19:35:06.927416 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.927082    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69cdff8dd4-d9l29" event={"ID":"cb6decc4-64ba-4211-b929-44e406c4dfca","Type":"ContainerDied","Data":"fdd3e1548aa018496e381496fa666a3ccea2182ce04bbf0f371f0bd914a3b78a"}
Jul 28 19:35:06.927416 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.927096    2572 scope.go:117] "RemoveContainer" containerID="cdd3a97e446f1ef25a264a34269b47f44b592af059031d974673d56e21ae8fec"
Jul 28 19:35:06.935203 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.935187    2572 scope.go:117] "RemoveContainer" containerID="cdd3a97e446f1ef25a264a34269b47f44b592af059031d974673d56e21ae8fec"
Jul 28 19:35:06.935487 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:35:06.935469    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdd3a97e446f1ef25a264a34269b47f44b592af059031d974673d56e21ae8fec\": container with ID starting with cdd3a97e446f1ef25a264a34269b47f44b592af059031d974673d56e21ae8fec not found: ID does not exist" containerID="cdd3a97e446f1ef25a264a34269b47f44b592af059031d974673d56e21ae8fec"
Jul 28 19:35:06.935551 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.935497    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdd3a97e446f1ef25a264a34269b47f44b592af059031d974673d56e21ae8fec"} err="failed to get container status \"cdd3a97e446f1ef25a264a34269b47f44b592af059031d974673d56e21ae8fec\": rpc error: code = NotFound desc = could not find container \"cdd3a97e446f1ef25a264a34269b47f44b592af059031d974673d56e21ae8fec\": container with ID starting with cdd3a97e446f1ef25a264a34269b47f44b592af059031d974673d56e21ae8fec not found: ID does not exist"
Jul 28 19:35:06.942213 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.942190    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69cdff8dd4-d9l29"]
Jul 28 19:35:06.946020 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:06.945998    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-69cdff8dd4-d9l29"]
Jul 28 19:35:08.863060 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:08.863020    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb6decc4-64ba-4211-b929-44e406c4dfca" path="/var/lib/kubelet/pods/cb6decc4-64ba-4211-b929-44e406c4dfca/volumes"
Jul 28 19:35:11.931955 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:11.931894    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-55f57c4956-sxppv" podUID="01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7" containerName="console" containerID="cri-o://e7de0eab59c24422fad579b96eec607c6def225f734b3254670ccf490ffd93a4" gracePeriod=15
Jul 28 19:35:12.168864 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.168839    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:35:12.227189 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.227132    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-service-ca\") pod \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") "
Jul 28 19:35:12.227189 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.227173    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mkqb\" (UniqueName: \"kubernetes.io/projected/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-kube-api-access-2mkqb\") pod \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") "
Jul 28 19:35:12.227336 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.227213    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-trusted-ca-bundle\") pod \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") "
Jul 28 19:35:12.227336 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.227249    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-console-serving-cert\") pod \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") "
Jul 28 19:35:12.227336 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.227267    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-console-config\") pod \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") "
Jul 28 19:35:12.227475 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.227378    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-oauth-serving-cert\") pod \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") "
Jul 28 19:35:12.227475 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.227419    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-console-oauth-config\") pod \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\" (UID: \"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7\") "
Jul 28 19:35:12.227623 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.227560    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-service-ca" (OuterVolumeSpecName: "service-ca") pod "01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7" (UID: "01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:35:12.227765 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.227737    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-console-config" (OuterVolumeSpecName: "console-config") pod "01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7" (UID: "01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:35:12.227885 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.227774    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7" (UID: "01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:35:12.227885 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.227817    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-service-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:12.227885 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.227846    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7" (UID: "01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:35:12.229228 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.229206    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7" (UID: "01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:35:12.229388 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.229370    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-kube-api-access-2mkqb" (OuterVolumeSpecName: "kube-api-access-2mkqb") pod "01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7" (UID: "01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7"). InnerVolumeSpecName "kube-api-access-2mkqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:35:12.229432 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.229382    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7" (UID: "01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:35:12.328901 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.328879    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mkqb\" (UniqueName: \"kubernetes.io/projected/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-kube-api-access-2mkqb\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:12.328901 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.328899    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-trusted-ca-bundle\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:12.329013 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.328908    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-console-serving-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:12.329013 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.328917    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-console-config\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:12.329013 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.328925    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-oauth-serving-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:12.329013 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.328932    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7-console-oauth-config\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:12.952912 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.952882    2572 generic.go:356] "Generic (PLEG): container finished" podID="01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7" containerID="e7de0eab59c24422fad579b96eec607c6def225f734b3254670ccf490ffd93a4" exitCode=0
Jul 28 19:35:12.953245 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.952926    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55f57c4956-sxppv"
Jul 28 19:35:12.953245 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.952940    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55f57c4956-sxppv" event={"ID":"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7","Type":"ContainerDied","Data":"e7de0eab59c24422fad579b96eec607c6def225f734b3254670ccf490ffd93a4"}
Jul 28 19:35:12.953245 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.952984    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55f57c4956-sxppv" event={"ID":"01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7","Type":"ContainerDied","Data":"05c17c3492060771a3226b8a2f173a5c94e1c29e46d16c0ef5a547a844e459ec"}
Jul 28 19:35:12.953245 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.953007    2572 scope.go:117] "RemoveContainer" containerID="e7de0eab59c24422fad579b96eec607c6def225f734b3254670ccf490ffd93a4"
Jul 28 19:35:12.961620 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.961600    2572 scope.go:117] "RemoveContainer" containerID="e7de0eab59c24422fad579b96eec607c6def225f734b3254670ccf490ffd93a4"
Jul 28 19:35:12.961905 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:35:12.961883    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7de0eab59c24422fad579b96eec607c6def225f734b3254670ccf490ffd93a4\": container with ID starting with e7de0eab59c24422fad579b96eec607c6def225f734b3254670ccf490ffd93a4 not found: ID does not exist" containerID="e7de0eab59c24422fad579b96eec607c6def225f734b3254670ccf490ffd93a4"
Jul 28 19:35:12.961965 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.961913    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7de0eab59c24422fad579b96eec607c6def225f734b3254670ccf490ffd93a4"} err="failed to get container status \"e7de0eab59c24422fad579b96eec607c6def225f734b3254670ccf490ffd93a4\": rpc error: code = NotFound desc = could not find container \"e7de0eab59c24422fad579b96eec607c6def225f734b3254670ccf490ffd93a4\": container with ID starting with e7de0eab59c24422fad579b96eec607c6def225f734b3254670ccf490ffd93a4 not found: ID does not exist"
Jul 28 19:35:12.968453 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.968428    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55f57c4956-sxppv"]
Jul 28 19:35:12.971625 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:12.971601    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55f57c4956-sxppv"]
Jul 28 19:35:14.861931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:14.861899    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7" path="/var/lib/kubelet/pods/01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7/volumes"
Jul 28 19:35:22.950947 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:22.950897    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-d67fbcb98-bhgfb" podUID="db388a9d-815d-415d-a3a2-14aafd18ce4b" containerName="console" containerID="cri-o://29cd45b7ba012d5366bce4c6c25afae18ae3184ba187e6d9b80182c1085146e1" gracePeriod=15
Jul 28 19:35:23.191243 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.191223    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:35:23.299390 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.299340    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-service-ca\") pod \"db388a9d-815d-415d-a3a2-14aafd18ce4b\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") "
Jul 28 19:35:23.299390 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.299369    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-trusted-ca-bundle\") pod \"db388a9d-815d-415d-a3a2-14aafd18ce4b\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") "
Jul 28 19:35:23.299390 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.299390    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffv9q\" (UniqueName: \"kubernetes.io/projected/db388a9d-815d-415d-a3a2-14aafd18ce4b-kube-api-access-ffv9q\") pod \"db388a9d-815d-415d-a3a2-14aafd18ce4b\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") "
Jul 28 19:35:23.299608 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.299427    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-console-config\") pod \"db388a9d-815d-415d-a3a2-14aafd18ce4b\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") "
Jul 28 19:35:23.299608 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.299476    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-oauth-serving-cert\") pod \"db388a9d-815d-415d-a3a2-14aafd18ce4b\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") "
Jul 28 19:35:23.299608 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.299493    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/db388a9d-815d-415d-a3a2-14aafd18ce4b-console-serving-cert\") pod \"db388a9d-815d-415d-a3a2-14aafd18ce4b\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") "
Jul 28 19:35:23.299608 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.299518    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/db388a9d-815d-415d-a3a2-14aafd18ce4b-console-oauth-config\") pod \"db388a9d-815d-415d-a3a2-14aafd18ce4b\" (UID: \"db388a9d-815d-415d-a3a2-14aafd18ce4b\") "
Jul 28 19:35:23.299844 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.299804    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-service-ca" (OuterVolumeSpecName: "service-ca") pod "db388a9d-815d-415d-a3a2-14aafd18ce4b" (UID: "db388a9d-815d-415d-a3a2-14aafd18ce4b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:35:23.299942 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.299863    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "db388a9d-815d-415d-a3a2-14aafd18ce4b" (UID: "db388a9d-815d-415d-a3a2-14aafd18ce4b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:35:23.299942 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.299873    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "db388a9d-815d-415d-a3a2-14aafd18ce4b" (UID: "db388a9d-815d-415d-a3a2-14aafd18ce4b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:35:23.299942 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.299915    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-console-config" (OuterVolumeSpecName: "console-config") pod "db388a9d-815d-415d-a3a2-14aafd18ce4b" (UID: "db388a9d-815d-415d-a3a2-14aafd18ce4b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:35:23.301560 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.301530    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db388a9d-815d-415d-a3a2-14aafd18ce4b-kube-api-access-ffv9q" (OuterVolumeSpecName: "kube-api-access-ffv9q") pod "db388a9d-815d-415d-a3a2-14aafd18ce4b" (UID: "db388a9d-815d-415d-a3a2-14aafd18ce4b"). InnerVolumeSpecName "kube-api-access-ffv9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:35:23.301560 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.301538    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db388a9d-815d-415d-a3a2-14aafd18ce4b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "db388a9d-815d-415d-a3a2-14aafd18ce4b" (UID: "db388a9d-815d-415d-a3a2-14aafd18ce4b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:35:23.301668 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.301552    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db388a9d-815d-415d-a3a2-14aafd18ce4b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "db388a9d-815d-415d-a3a2-14aafd18ce4b" (UID: "db388a9d-815d-415d-a3a2-14aafd18ce4b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:35:23.399955 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.399936    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-console-config\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:23.399955 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.399953    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-oauth-serving-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:23.400057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.399962    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/db388a9d-815d-415d-a3a2-14aafd18ce4b-console-serving-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:23.400057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.399970    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/db388a9d-815d-415d-a3a2-14aafd18ce4b-console-oauth-config\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:23.400057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.399977    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-service-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:23.400057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.399985    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db388a9d-815d-415d-a3a2-14aafd18ce4b-trusted-ca-bundle\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:23.400057 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.399992    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffv9q\" (UniqueName: \"kubernetes.io/projected/db388a9d-815d-415d-a3a2-14aafd18ce4b-kube-api-access-ffv9q\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:35:23.990481 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.990457    2572 generic.go:356] "Generic (PLEG): container finished" podID="db388a9d-815d-415d-a3a2-14aafd18ce4b" containerID="29cd45b7ba012d5366bce4c6c25afae18ae3184ba187e6d9b80182c1085146e1" exitCode=0
Jul 28 19:35:23.990810 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.990495    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d67fbcb98-bhgfb"
Jul 28 19:35:23.990810 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.990520    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d67fbcb98-bhgfb" event={"ID":"db388a9d-815d-415d-a3a2-14aafd18ce4b","Type":"ContainerDied","Data":"29cd45b7ba012d5366bce4c6c25afae18ae3184ba187e6d9b80182c1085146e1"}
Jul 28 19:35:23.990810 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.990553    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d67fbcb98-bhgfb" event={"ID":"db388a9d-815d-415d-a3a2-14aafd18ce4b","Type":"ContainerDied","Data":"0873ce18030abba0944c736f2bca25138f7071734abcdc37e186b99bc28b6cad"}
Jul 28 19:35:23.990810 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.990572    2572 scope.go:117] "RemoveContainer" containerID="29cd45b7ba012d5366bce4c6c25afae18ae3184ba187e6d9b80182c1085146e1"
Jul 28 19:35:23.998476 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.998451    2572 scope.go:117] "RemoveContainer" containerID="29cd45b7ba012d5366bce4c6c25afae18ae3184ba187e6d9b80182c1085146e1"
Jul 28 19:35:23.998702 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:35:23.998685    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29cd45b7ba012d5366bce4c6c25afae18ae3184ba187e6d9b80182c1085146e1\": container with ID starting with 29cd45b7ba012d5366bce4c6c25afae18ae3184ba187e6d9b80182c1085146e1 not found: ID does not exist" containerID="29cd45b7ba012d5366bce4c6c25afae18ae3184ba187e6d9b80182c1085146e1"
Jul 28 19:35:23.998752 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:23.998709    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29cd45b7ba012d5366bce4c6c25afae18ae3184ba187e6d9b80182c1085146e1"} err="failed to get container status \"29cd45b7ba012d5366bce4c6c25afae18ae3184ba187e6d9b80182c1085146e1\": rpc error: code = NotFound desc = could not find container \"29cd45b7ba012d5366bce4c6c25afae18ae3184ba187e6d9b80182c1085146e1\": container with ID starting with 29cd45b7ba012d5366bce4c6c25afae18ae3184ba187e6d9b80182c1085146e1 not found: ID does not exist"
Jul 28 19:35:24.009287 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:24.009269    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d67fbcb98-bhgfb"]
Jul 28 19:35:24.012646 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:24.012629    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d67fbcb98-bhgfb"]
Jul 28 19:35:24.861904 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:24.861881    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db388a9d-815d-415d-a3a2-14aafd18ce4b" path="/var/lib/kubelet/pods/db388a9d-815d-415d-a3a2-14aafd18ce4b/volumes"
Jul 28 19:35:28.729375 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:28.729287    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:28.834658 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:28.834632    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:28.930271 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:28.930248    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:48.667202 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:48.667159    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs\") pod \"network-metrics-daemon-8b8d6\" (UID: \"303f736b-24a3-4e9b-8904-e4fde37ab7e0\") " pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:35:48.669641 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:48.669611    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303f736b-24a3-4e9b-8904-e4fde37ab7e0-metrics-certs\") pod \"network-metrics-daemon-8b8d6\" (UID: \"303f736b-24a3-4e9b-8904-e4fde37ab7e0\") " pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:35:48.961926 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:48.961867    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zz5d7\""
Jul 28 19:35:48.969909 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:48.969886    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8b8d6"
Jul 28 19:35:49.092934 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:49.092906    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8b8d6"]
Jul 28 19:35:49.095863 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:35:49.095833    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod303f736b_24a3_4e9b_8904_e4fde37ab7e0.slice/crio-0664edffd4c4702d9c4693bd0daf9d84edafee7fa79cafabb3f06aa1287d2890 WatchSource:0}: Error finding container 0664edffd4c4702d9c4693bd0daf9d84edafee7fa79cafabb3f06aa1287d2890: Status 404 returned error can't find the container with id 0664edffd4c4702d9c4693bd0daf9d84edafee7fa79cafabb3f06aa1287d2890
Jul 28 19:35:50.081594 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:50.081560    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8b8d6" event={"ID":"303f736b-24a3-4e9b-8904-e4fde37ab7e0","Type":"ContainerStarted","Data":"0520a170a52875b4263b37f8e0ea20b9d99106710b2c03d34808ba16f86ff626"}
Jul 28 19:35:50.081886 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:50.081619    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8b8d6" event={"ID":"303f736b-24a3-4e9b-8904-e4fde37ab7e0","Type":"ContainerStarted","Data":"0664edffd4c4702d9c4693bd0daf9d84edafee7fa79cafabb3f06aa1287d2890"}
Jul 28 19:35:51.086864 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:51.086830    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8b8d6" event={"ID":"303f736b-24a3-4e9b-8904-e4fde37ab7e0","Type":"ContainerStarted","Data":"7862684b1edf552bc163fd5b562f764b63d34294de397f7c34179cf08a01d89b"}
Jul 28 19:35:51.102946 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:51.102894    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8b8d6" podStartSLOduration=375.211461102 podStartE2EDuration="6m16.102877092s" totalImagesPullingTime="891.41599ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:29:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:51.101161439 +0000 UTC m=+376.688824384" watchObservedRunningTime="2026-07-28 19:35:51.102877092 +0000 UTC m=+376.690540036"
Jul 28 19:35:51.183735 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:51.183702    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:51.234082 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:51.234041    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:51.290060 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:51.290036    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:52.158843 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:52.158816    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:52.211991 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:52.211971    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:58.153907 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:58.153873    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:58.215040 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:35:58.215006    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:04.056289 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:04.056255    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:04.156694 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:04.156666    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:06.648835 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.648803    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55b95f6b68-rs9zz"]
Jul 28 19:36:06.649197 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.649040    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb6decc4-64ba-4211-b929-44e406c4dfca" containerName="console"
Jul 28 19:36:06.649197 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.649050    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb6decc4-64ba-4211-b929-44e406c4dfca" containerName="console"
Jul 28 19:36:06.649197 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.649060    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db388a9d-815d-415d-a3a2-14aafd18ce4b" containerName="console"
Jul 28 19:36:06.649197 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.649088    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="db388a9d-815d-415d-a3a2-14aafd18ce4b" containerName="console"
Jul 28 19:36:06.649197 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.649108    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f427e901-fd2a-4d35-a853-fe83fabbca35" containerName="registry"
Jul 28 19:36:06.649197 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.649113    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f427e901-fd2a-4d35-a853-fe83fabbca35" containerName="registry"
Jul 28 19:36:06.649197 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.649121    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7" containerName="console"
Jul 28 19:36:06.649197 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.649126    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7" containerName="console"
Jul 28 19:36:06.649197 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.649166    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f427e901-fd2a-4d35-a853-fe83fabbca35" containerName="registry"
Jul 28 19:36:06.649197 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.649173    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb6decc4-64ba-4211-b929-44e406c4dfca" containerName="console"
Jul 28 19:36:06.649197 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.649180    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="01e3f3d7-8b49-4df7-bde0-d8ed7252c9d7" containerName="console"
Jul 28 19:36:06.649197 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.649186    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="db388a9d-815d-415d-a3a2-14aafd18ce4b" containerName="console"
Jul 28 19:36:06.651994 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.651973    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.662549 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.662525    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55b95f6b68-rs9zz"]
Jul 28 19:36:06.728013 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.727985    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:06.776035 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.776014    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:06.784563 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.784542    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f916ea54-9231-4f6b-a810-a6a98a8fcaae-console-oauth-config\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.784647 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.784567    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-service-ca\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.784647 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.784589    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-trusted-ca-bundle\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.784647 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.784616    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-console-config\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.784743 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.784657    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7bcv\" (UniqueName: \"kubernetes.io/projected/f916ea54-9231-4f6b-a810-a6a98a8fcaae-kube-api-access-h7bcv\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.784743 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.784705    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f916ea54-9231-4f6b-a810-a6a98a8fcaae-console-serving-cert\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.784799 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.784739    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-oauth-serving-cert\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.885213 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.885186    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-console-config\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.885328 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.885221    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7bcv\" (UniqueName: \"kubernetes.io/projected/f916ea54-9231-4f6b-a810-a6a98a8fcaae-kube-api-access-h7bcv\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.885328 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.885247    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f916ea54-9231-4f6b-a810-a6a98a8fcaae-console-serving-cert\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.885328 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.885278    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-oauth-serving-cert\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.885328 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.885317    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f916ea54-9231-4f6b-a810-a6a98a8fcaae-console-oauth-config\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.885499 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.885363    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-service-ca\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.885499 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.885409    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-trusted-ca-bundle\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.885962 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.885930    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-console-config\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.886042 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.886012    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-oauth-serving-cert\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.886128 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.886096    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-service-ca\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.886190 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.886166    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-trusted-ca-bundle\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.887728 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.887707    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f916ea54-9231-4f6b-a810-a6a98a8fcaae-console-serving-cert\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.887831 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.887816    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f916ea54-9231-4f6b-a810-a6a98a8fcaae-console-oauth-config\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.892166 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.892139    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7bcv\" (UniqueName: \"kubernetes.io/projected/f916ea54-9231-4f6b-a810-a6a98a8fcaae-kube-api-access-h7bcv\") pod \"console-55b95f6b68-rs9zz\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") " pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:06.963722 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:06.963669    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:07.087911 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:07.087885    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55b95f6b68-rs9zz"]
Jul 28 19:36:07.090163 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:36:07.090135    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf916ea54_9231_4f6b_a810_a6a98a8fcaae.slice/crio-6a3ad15dbb285c9fd3a2ce563f953acfb3ab567aad37b63d2d4f8125d7a50647 WatchSource:0}: Error finding container 6a3ad15dbb285c9fd3a2ce563f953acfb3ab567aad37b63d2d4f8125d7a50647: Status 404 returned error can't find the container with id 6a3ad15dbb285c9fd3a2ce563f953acfb3ab567aad37b63d2d4f8125d7a50647
Jul 28 19:36:07.145772 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:07.145749    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55b95f6b68-rs9zz" event={"ID":"f916ea54-9231-4f6b-a810-a6a98a8fcaae","Type":"ContainerStarted","Data":"6a3ad15dbb285c9fd3a2ce563f953acfb3ab567aad37b63d2d4f8125d7a50647"}
Jul 28 19:36:08.151297 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:08.151255    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55b95f6b68-rs9zz" event={"ID":"f916ea54-9231-4f6b-a810-a6a98a8fcaae","Type":"ContainerStarted","Data":"1a3963f023827db201c5eb27325c45c0a3f5ce78ca3070f3d2abc7c25cbddbe1"}
Jul 28 19:36:08.168214 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:08.168158    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55b95f6b68-rs9zz" podStartSLOduration=2.168137713 podStartE2EDuration="2.168137713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:08.167767778 +0000 UTC m=+393.755430724" watchObservedRunningTime="2026-07-28 19:36:08.168137713 +0000 UTC m=+393.755800649"
Jul 28 19:36:16.964303 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:16.964266    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:16.964769 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:16.964329    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:16.969262 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:16.969235    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:17.186828 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:17.186808    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:36:17.232414 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:17.232350    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5556f97b8b-dnc78"]
Jul 28 19:36:17.330234 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:17.330192    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:17.385173 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:17.385148    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:17.433020 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:17.433000    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:21.356510 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:21.356477    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:21.454961 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:21.454932    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:21.503406 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:21.503374    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:24.754815 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:24.754778    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:24.855548 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:24.855512    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:40.411337 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:40.411302    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:42.255547 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.255490    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5556f97b8b-dnc78" podUID="aed855e7-36d0-4bbc-b233-7c93dcb687dd" containerName="console" containerID="cri-o://4ba664f0d928ec5a938d96630fee513cba99a71e72fb9442f84d47bd6a80a47d" gracePeriod=15
Jul 28 19:36:42.484029 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.484007    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:36:42.632375 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.632348    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-trusted-ca-bundle\") pod \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") "
Jul 28 19:36:42.632502 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.632380    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-oauth-serving-cert\") pod \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") "
Jul 28 19:36:42.632502 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.632401    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aed855e7-36d0-4bbc-b233-7c93dcb687dd-console-oauth-config\") pod \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") "
Jul 28 19:36:42.632502 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.632424    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aed855e7-36d0-4bbc-b233-7c93dcb687dd-console-serving-cert\") pod \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") "
Jul 28 19:36:42.632669 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.632517    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-service-ca\") pod \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") "
Jul 28 19:36:42.632669 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.632566    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hczlx\" (UniqueName: \"kubernetes.io/projected/aed855e7-36d0-4bbc-b233-7c93dcb687dd-kube-api-access-hczlx\") pod \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") "
Jul 28 19:36:42.632669 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.632611    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-console-config\") pod \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\" (UID: \"aed855e7-36d0-4bbc-b233-7c93dcb687dd\") "
Jul 28 19:36:42.632798 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.632748    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "aed855e7-36d0-4bbc-b233-7c93dcb687dd" (UID: "aed855e7-36d0-4bbc-b233-7c93dcb687dd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:36:42.632847 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.632829    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "aed855e7-36d0-4bbc-b233-7c93dcb687dd" (UID: "aed855e7-36d0-4bbc-b233-7c93dcb687dd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:36:42.632993 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.632972    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-oauth-serving-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:36:42.633053 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.633006    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-console-config" (OuterVolumeSpecName: "console-config") pod "aed855e7-36d0-4bbc-b233-7c93dcb687dd" (UID: "aed855e7-36d0-4bbc-b233-7c93dcb687dd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:36:42.633265 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.633243    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-service-ca" (OuterVolumeSpecName: "service-ca") pod "aed855e7-36d0-4bbc-b233-7c93dcb687dd" (UID: "aed855e7-36d0-4bbc-b233-7c93dcb687dd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:36:42.634724 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.634689    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aed855e7-36d0-4bbc-b233-7c93dcb687dd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "aed855e7-36d0-4bbc-b233-7c93dcb687dd" (UID: "aed855e7-36d0-4bbc-b233-7c93dcb687dd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:36:42.634830 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.634755    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aed855e7-36d0-4bbc-b233-7c93dcb687dd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "aed855e7-36d0-4bbc-b233-7c93dcb687dd" (UID: "aed855e7-36d0-4bbc-b233-7c93dcb687dd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:36:42.634830 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.634756    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aed855e7-36d0-4bbc-b233-7c93dcb687dd-kube-api-access-hczlx" (OuterVolumeSpecName: "kube-api-access-hczlx") pod "aed855e7-36d0-4bbc-b233-7c93dcb687dd" (UID: "aed855e7-36d0-4bbc-b233-7c93dcb687dd"). InnerVolumeSpecName "kube-api-access-hczlx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:36:42.734055 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.734030    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-console-config\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:36:42.734055 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.734050    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-trusted-ca-bundle\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:36:42.734196 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.734060    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aed855e7-36d0-4bbc-b233-7c93dcb687dd-console-oauth-config\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:36:42.734196 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.734084    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aed855e7-36d0-4bbc-b233-7c93dcb687dd-console-serving-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:36:42.734196 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.734093    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aed855e7-36d0-4bbc-b233-7c93dcb687dd-service-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:36:42.734196 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:42.734101    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hczlx\" (UniqueName: \"kubernetes.io/projected/aed855e7-36d0-4bbc-b233-7c93dcb687dd-kube-api-access-hczlx\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:36:43.273803 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:43.273770    2572 generic.go:356] "Generic (PLEG): container finished" podID="aed855e7-36d0-4bbc-b233-7c93dcb687dd" containerID="4ba664f0d928ec5a938d96630fee513cba99a71e72fb9442f84d47bd6a80a47d" exitCode=0
Jul 28 19:36:43.274146 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:43.273821    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5556f97b8b-dnc78"
Jul 28 19:36:43.274146 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:43.273860    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5556f97b8b-dnc78" event={"ID":"aed855e7-36d0-4bbc-b233-7c93dcb687dd","Type":"ContainerDied","Data":"4ba664f0d928ec5a938d96630fee513cba99a71e72fb9442f84d47bd6a80a47d"}
Jul 28 19:36:43.274146 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:43.273913    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5556f97b8b-dnc78" event={"ID":"aed855e7-36d0-4bbc-b233-7c93dcb687dd","Type":"ContainerDied","Data":"b0e1d10f48825d0b7fffecaddf360ea162ca83e8793226ee3a593dfdafd98535"}
Jul 28 19:36:43.274146 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:43.273941    2572 scope.go:117] "RemoveContainer" containerID="4ba664f0d928ec5a938d96630fee513cba99a71e72fb9442f84d47bd6a80a47d"
Jul 28 19:36:43.283182 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:43.283162    2572 scope.go:117] "RemoveContainer" containerID="4ba664f0d928ec5a938d96630fee513cba99a71e72fb9442f84d47bd6a80a47d"
Jul 28 19:36:43.283493 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:36:43.283459    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ba664f0d928ec5a938d96630fee513cba99a71e72fb9442f84d47bd6a80a47d\": container with ID starting with 4ba664f0d928ec5a938d96630fee513cba99a71e72fb9442f84d47bd6a80a47d not found: ID does not exist" containerID="4ba664f0d928ec5a938d96630fee513cba99a71e72fb9442f84d47bd6a80a47d"
Jul 28 19:36:43.283541 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:43.283500    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ba664f0d928ec5a938d96630fee513cba99a71e72fb9442f84d47bd6a80a47d"} err="failed to get container status \"4ba664f0d928ec5a938d96630fee513cba99a71e72fb9442f84d47bd6a80a47d\": rpc error: code = NotFound desc = could not find container \"4ba664f0d928ec5a938d96630fee513cba99a71e72fb9442f84d47bd6a80a47d\": container with ID starting with 4ba664f0d928ec5a938d96630fee513cba99a71e72fb9442f84d47bd6a80a47d not found: ID does not exist"
Jul 28 19:36:43.289300 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:43.289259    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5556f97b8b-dnc78"]
Jul 28 19:36:43.291241 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:43.291220    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5556f97b8b-dnc78"]
Jul 28 19:36:44.863199 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:36:44.863168    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aed855e7-36d0-4bbc-b233-7c93dcb687dd" path="/var/lib/kubelet/pods/aed855e7-36d0-4bbc-b233-7c93dcb687dd/volumes"
Jul 28 19:37:09.155848 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.155808    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:09.252047 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.252020    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:09.725017 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.724986    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm"]
Jul 28 19:37:09.725316 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.725301    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aed855e7-36d0-4bbc-b233-7c93dcb687dd" containerName="console"
Jul 28 19:37:09.725372 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.725318    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="aed855e7-36d0-4bbc-b233-7c93dcb687dd" containerName="console"
Jul 28 19:37:09.725408 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.725376    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="aed855e7-36d0-4bbc-b233-7c93dcb687dd" containerName="console"
Jul 28 19:37:09.728501 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.728482    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm"
Jul 28 19:37:09.731035 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.731012    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 19:37:09.732047 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.732026    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-whzjq\""
Jul 28 19:37:09.732158 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.732031    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 19:37:09.737246 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.737219    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm"]
Jul 28 19:37:09.803523 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.803496    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l299q\" (UniqueName: \"kubernetes.io/projected/2819af76-1171-4ebd-8784-8287957f96d2-kube-api-access-l299q\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm\" (UID: \"2819af76-1171-4ebd-8784-8287957f96d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm"
Jul 28 19:37:09.803631 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.803538    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2819af76-1171-4ebd-8784-8287957f96d2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm\" (UID: \"2819af76-1171-4ebd-8784-8287957f96d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm"
Jul 28 19:37:09.803631 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.803565    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2819af76-1171-4ebd-8784-8287957f96d2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm\" (UID: \"2819af76-1171-4ebd-8784-8287957f96d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm"
Jul 28 19:37:09.904654 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.904625    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2819af76-1171-4ebd-8784-8287957f96d2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm\" (UID: \"2819af76-1171-4ebd-8784-8287957f96d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm"
Jul 28 19:37:09.904776 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.904670    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2819af76-1171-4ebd-8784-8287957f96d2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm\" (UID: \"2819af76-1171-4ebd-8784-8287957f96d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm"
Jul 28 19:37:09.904842 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.904814    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l299q\" (UniqueName: \"kubernetes.io/projected/2819af76-1171-4ebd-8784-8287957f96d2-kube-api-access-l299q\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm\" (UID: \"2819af76-1171-4ebd-8784-8287957f96d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm"
Jul 28 19:37:09.905006 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.904981    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2819af76-1171-4ebd-8784-8287957f96d2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm\" (UID: \"2819af76-1171-4ebd-8784-8287957f96d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm"
Jul 28 19:37:09.905102 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.905033    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2819af76-1171-4ebd-8784-8287957f96d2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm\" (UID: \"2819af76-1171-4ebd-8784-8287957f96d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm"
Jul 28 19:37:09.912976 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:09.912949    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l299q\" (UniqueName: \"kubernetes.io/projected/2819af76-1171-4ebd-8784-8287957f96d2-kube-api-access-l299q\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm\" (UID: \"2819af76-1171-4ebd-8784-8287957f96d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm"
Jul 28 19:37:10.039231 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:10.039180    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm"
Jul 28 19:37:10.158954 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:10.158928    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm"]
Jul 28 19:37:10.162227 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:37:10.162203    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2819af76_1171_4ebd_8784_8287957f96d2.slice/crio-53a65f11f31133a48a10e87057defb7c762b2507a3a91d1e6b7e1fcb9a710f99 WatchSource:0}: Error finding container 53a65f11f31133a48a10e87057defb7c762b2507a3a91d1e6b7e1fcb9a710f99: Status 404 returned error can't find the container with id 53a65f11f31133a48a10e87057defb7c762b2507a3a91d1e6b7e1fcb9a710f99
Jul 28 19:37:10.368932 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:10.368907    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm" event={"ID":"2819af76-1171-4ebd-8784-8287957f96d2","Type":"ContainerStarted","Data":"53a65f11f31133a48a10e87057defb7c762b2507a3a91d1e6b7e1fcb9a710f99"}
Jul 28 19:37:17.396952 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:17.396922    2572 generic.go:356] "Generic (PLEG): container finished" podID="2819af76-1171-4ebd-8784-8287957f96d2" containerID="d2d8f7d43bb0a8d6fda175861658cfda3f3a616010df1cfcaef71c051f19ee9a" exitCode=0
Jul 28 19:37:17.397317 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:17.396959    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm" event={"ID":"2819af76-1171-4ebd-8784-8287957f96d2","Type":"ContainerDied","Data":"d2d8f7d43bb0a8d6fda175861658cfda3f3a616010df1cfcaef71c051f19ee9a"}
Jul 28 19:37:20.409931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:20.409903    2572 generic.go:356] "Generic (PLEG): container finished" podID="2819af76-1171-4ebd-8784-8287957f96d2" containerID="6be5d64c4bc205a4a70c5bf2d3952e598508dffb00aea9a6c1857643a0a8f623" exitCode=0
Jul 28 19:37:20.410295 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:20.409964    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm" event={"ID":"2819af76-1171-4ebd-8784-8287957f96d2","Type":"ContainerDied","Data":"6be5d64c4bc205a4a70c5bf2d3952e598508dffb00aea9a6c1857643a0a8f623"}
Jul 28 19:37:27.441461 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:27.441426    2572 generic.go:356] "Generic (PLEG): container finished" podID="2819af76-1171-4ebd-8784-8287957f96d2" containerID="adfbf6a5d98bf65e1864cb89d11c383a14a1771e85fcc50250389536f9f1b5e5" exitCode=0
Jul 28 19:37:27.441890 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:27.441516    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm" event={"ID":"2819af76-1171-4ebd-8784-8287957f96d2","Type":"ContainerDied","Data":"adfbf6a5d98bf65e1864cb89d11c383a14a1771e85fcc50250389536f9f1b5e5"}
Jul 28 19:37:28.566932 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:28.566912    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm"
Jul 28 19:37:28.654281 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:28.654260    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l299q\" (UniqueName: \"kubernetes.io/projected/2819af76-1171-4ebd-8784-8287957f96d2-kube-api-access-l299q\") pod \"2819af76-1171-4ebd-8784-8287957f96d2\" (UID: \"2819af76-1171-4ebd-8784-8287957f96d2\") "
Jul 28 19:37:28.654394 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:28.654323    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2819af76-1171-4ebd-8784-8287957f96d2-util\") pod \"2819af76-1171-4ebd-8784-8287957f96d2\" (UID: \"2819af76-1171-4ebd-8784-8287957f96d2\") "
Jul 28 19:37:28.654512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:28.654489    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2819af76-1171-4ebd-8784-8287957f96d2-bundle\") pod \"2819af76-1171-4ebd-8784-8287957f96d2\" (UID: \"2819af76-1171-4ebd-8784-8287957f96d2\") "
Jul 28 19:37:28.655179 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:28.655156    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2819af76-1171-4ebd-8784-8287957f96d2-bundle" (OuterVolumeSpecName: "bundle") pod "2819af76-1171-4ebd-8784-8287957f96d2" (UID: "2819af76-1171-4ebd-8784-8287957f96d2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:37:28.656440 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:28.656419    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2819af76-1171-4ebd-8784-8287957f96d2-kube-api-access-l299q" (OuterVolumeSpecName: "kube-api-access-l299q") pod "2819af76-1171-4ebd-8784-8287957f96d2" (UID: "2819af76-1171-4ebd-8784-8287957f96d2"). InnerVolumeSpecName "kube-api-access-l299q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:37:28.660554 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:28.660530    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2819af76-1171-4ebd-8784-8287957f96d2-util" (OuterVolumeSpecName: "util") pod "2819af76-1171-4ebd-8784-8287957f96d2" (UID: "2819af76-1171-4ebd-8784-8287957f96d2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:37:28.754984 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:28.754936    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2819af76-1171-4ebd-8784-8287957f96d2-util\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:37:28.754984 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:28.754956    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2819af76-1171-4ebd-8784-8287957f96d2-bundle\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:37:28.754984 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:28.754965    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l299q\" (UniqueName: \"kubernetes.io/projected/2819af76-1171-4ebd-8784-8287957f96d2-kube-api-access-l299q\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:37:29.450796 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:29.450780    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm"
Jul 28 19:37:29.450796 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:29.450783    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xpqxm" event={"ID":"2819af76-1171-4ebd-8784-8287957f96d2","Type":"ContainerDied","Data":"53a65f11f31133a48a10e87057defb7c762b2507a3a91d1e6b7e1fcb9a710f99"}
Jul 28 19:37:29.450930 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:29.450821    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="53a65f11f31133a48a10e87057defb7c762b2507a3a91d1e6b7e1fcb9a710f99"
Jul 28 19:37:30.694095 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:30.694053    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:31.483350 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:31.483311    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:32.660598 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.660556    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7sh4r"]
Jul 28 19:37:32.661177 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.661009    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2819af76-1171-4ebd-8784-8287957f96d2" containerName="util"
Jul 28 19:37:32.661177 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.661030    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2819af76-1171-4ebd-8784-8287957f96d2" containerName="util"
Jul 28 19:37:32.661177 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.661057    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2819af76-1171-4ebd-8784-8287957f96d2" containerName="pull"
Jul 28 19:37:32.661177 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.661086    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2819af76-1171-4ebd-8784-8287957f96d2" containerName="pull"
Jul 28 19:37:32.661177 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.661107    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2819af76-1171-4ebd-8784-8287957f96d2" containerName="extract"
Jul 28 19:37:32.661177 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.661115    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2819af76-1171-4ebd-8784-8287957f96d2" containerName="extract"
Jul 28 19:37:32.661533 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.661211    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2819af76-1171-4ebd-8784-8287957f96d2" containerName="extract"
Jul 28 19:37:32.664460 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.664435    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7sh4r"
Jul 28 19:37:32.667185 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.667158    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 28 19:37:32.667328 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.667276    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:37:32.667328 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.667287    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-8zsdq\""
Jul 28 19:37:32.676429 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.676404    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7sh4r"]
Jul 28 19:37:32.784064 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.784038    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg7s6\" (UniqueName: \"kubernetes.io/projected/ef84eccb-c5a7-4cbf-a897-dbfe919ed559-kube-api-access-fg7s6\") pod \"cert-manager-operator-controller-manager-55c444bf86-7sh4r\" (UID: \"ef84eccb-c5a7-4cbf-a897-dbfe919ed559\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7sh4r"
Jul 28 19:37:32.784206 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.784124    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef84eccb-c5a7-4cbf-a897-dbfe919ed559-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7sh4r\" (UID: \"ef84eccb-c5a7-4cbf-a897-dbfe919ed559\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7sh4r"
Jul 28 19:37:32.885188 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.885158    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg7s6\" (UniqueName: \"kubernetes.io/projected/ef84eccb-c5a7-4cbf-a897-dbfe919ed559-kube-api-access-fg7s6\") pod \"cert-manager-operator-controller-manager-55c444bf86-7sh4r\" (UID: \"ef84eccb-c5a7-4cbf-a897-dbfe919ed559\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7sh4r"
Jul 28 19:37:32.885297 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.885205    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef84eccb-c5a7-4cbf-a897-dbfe919ed559-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7sh4r\" (UID: \"ef84eccb-c5a7-4cbf-a897-dbfe919ed559\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7sh4r"
Jul 28 19:37:32.885525 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.885508    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef84eccb-c5a7-4cbf-a897-dbfe919ed559-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7sh4r\" (UID: \"ef84eccb-c5a7-4cbf-a897-dbfe919ed559\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7sh4r"
Jul 28 19:37:32.893026 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.893005    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg7s6\" (UniqueName: \"kubernetes.io/projected/ef84eccb-c5a7-4cbf-a897-dbfe919ed559-kube-api-access-fg7s6\") pod \"cert-manager-operator-controller-manager-55c444bf86-7sh4r\" (UID: \"ef84eccb-c5a7-4cbf-a897-dbfe919ed559\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7sh4r"
Jul 28 19:37:32.974634 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:32.974579    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7sh4r"
Jul 28 19:37:33.107493 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:33.107465    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7sh4r"]
Jul 28 19:37:33.109310 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:37:33.109278    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef84eccb_c5a7_4cbf_a897_dbfe919ed559.slice/crio-0e9c36754455aecff7909fcc3b5e4dc907657aaf1650d1dbf32feab1015f63dd WatchSource:0}: Error finding container 0e9c36754455aecff7909fcc3b5e4dc907657aaf1650d1dbf32feab1015f63dd: Status 404 returned error can't find the container with id 0e9c36754455aecff7909fcc3b5e4dc907657aaf1650d1dbf32feab1015f63dd
Jul 28 19:37:33.467593 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:33.467559    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7sh4r" event={"ID":"ef84eccb-c5a7-4cbf-a897-dbfe919ed559","Type":"ContainerStarted","Data":"0e9c36754455aecff7909fcc3b5e4dc907657aaf1650d1dbf32feab1015f63dd"}
Jul 28 19:37:34.510124 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:34.510093    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:34.610086 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:34.610029    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:35.478045 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:35.478013    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7sh4r" event={"ID":"ef84eccb-c5a7-4cbf-a897-dbfe919ed559","Type":"ContainerStarted","Data":"d7783bcddf9a197986a840055847e3a348799489b70a71f87e8c87759099e826"}
Jul 28 19:37:35.499368 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:35.499320    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7sh4r" podStartSLOduration=1.937007147 podStartE2EDuration="3.499300559s" totalImagesPullingTime="1.562293412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:37:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:35.497831343 +0000 UTC m=+481.085494289" watchObservedRunningTime="2026-07-28 19:37:35.499300559 +0000 UTC m=+481.086963505"
Jul 28 19:37:35.829385 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:35.829361    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:35.955417 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:35.955392    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:36.121301 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.121240    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4"]
Jul 28 19:37:36.125756 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.125737    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4"
Jul 28 19:37:36.128194 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.128174    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 19:37:36.128271 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.128174    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 19:37:36.128991 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.128977    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-whzjq\""
Jul 28 19:37:36.134491 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.134470    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4"]
Jul 28 19:37:36.214262 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.214239    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8143798c-8e3c-4a3b-a14f-af4a15940521-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4\" (UID: \"8143798c-8e3c-4a3b-a14f-af4a15940521\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4"
Jul 28 19:37:36.214358 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.214281    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv8hb\" (UniqueName: \"kubernetes.io/projected/8143798c-8e3c-4a3b-a14f-af4a15940521-kube-api-access-nv8hb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4\" (UID: \"8143798c-8e3c-4a3b-a14f-af4a15940521\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4"
Jul 28 19:37:36.214358 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.214304    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8143798c-8e3c-4a3b-a14f-af4a15940521-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4\" (UID: \"8143798c-8e3c-4a3b-a14f-af4a15940521\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4"
Jul 28 19:37:36.314889 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.314865    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv8hb\" (UniqueName: \"kubernetes.io/projected/8143798c-8e3c-4a3b-a14f-af4a15940521-kube-api-access-nv8hb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4\" (UID: \"8143798c-8e3c-4a3b-a14f-af4a15940521\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4"
Jul 28 19:37:36.314993 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.314897    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8143798c-8e3c-4a3b-a14f-af4a15940521-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4\" (UID: \"8143798c-8e3c-4a3b-a14f-af4a15940521\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4"
Jul 28 19:37:36.314993 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.314935    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8143798c-8e3c-4a3b-a14f-af4a15940521-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4\" (UID: \"8143798c-8e3c-4a3b-a14f-af4a15940521\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4"
Jul 28 19:37:36.315268 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.315249    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8143798c-8e3c-4a3b-a14f-af4a15940521-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4\" (UID: \"8143798c-8e3c-4a3b-a14f-af4a15940521\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4"
Jul 28 19:37:36.315338 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.315318    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8143798c-8e3c-4a3b-a14f-af4a15940521-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4\" (UID: \"8143798c-8e3c-4a3b-a14f-af4a15940521\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4"
Jul 28 19:37:36.322988 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.322964    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv8hb\" (UniqueName: \"kubernetes.io/projected/8143798c-8e3c-4a3b-a14f-af4a15940521-kube-api-access-nv8hb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4\" (UID: \"8143798c-8e3c-4a3b-a14f-af4a15940521\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4"
Jul 28 19:37:36.434631 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.434577    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4"
Jul 28 19:37:36.461019 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.460985    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:36.573865 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.573836    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:36.590745 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:36.590724    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4"]
Jul 28 19:37:36.593580 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:37:36.593557    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8143798c_8e3c_4a3b_a14f_af4a15940521.slice/crio-836cf28160e43e7d51961756da4cc1c19f679201ae83b62526210c06203289c2 WatchSource:0}: Error finding container 836cf28160e43e7d51961756da4cc1c19f679201ae83b62526210c06203289c2: Status 404 returned error can't find the container with id 836cf28160e43e7d51961756da4cc1c19f679201ae83b62526210c06203289c2
Jul 28 19:37:37.491710 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:37.491680    2572 generic.go:356] "Generic (PLEG): container finished" podID="8143798c-8e3c-4a3b-a14f-af4a15940521" containerID="2c6d80c61f5d7c0211ee0f3c96a21b048a774510423d2303dafd40a61e2e63e5" exitCode=0
Jul 28 19:37:37.492089 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:37.491778    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4" event={"ID":"8143798c-8e3c-4a3b-a14f-af4a15940521","Type":"ContainerDied","Data":"2c6d80c61f5d7c0211ee0f3c96a21b048a774510423d2303dafd40a61e2e63e5"}
Jul 28 19:37:37.492089 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:37.491831    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4" event={"ID":"8143798c-8e3c-4a3b-a14f-af4a15940521","Type":"ContainerStarted","Data":"836cf28160e43e7d51961756da4cc1c19f679201ae83b62526210c06203289c2"}
Jul 28 19:37:38.476839 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:38.476804    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:38.859019 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:38.858983    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:39.056030 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:39.056001    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-mwkr7"]
Jul 28 19:37:39.064809 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:39.064784    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwkr7"
Jul 28 19:37:39.067254 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:39.067202    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 19:37:39.067254 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:39.067235    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-89sch\""
Jul 28 19:37:39.068655 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:39.068463    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 19:37:39.070300 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:39.070280    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-mwkr7"]
Jul 28 19:37:39.139197 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:39.139135    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b70073f8-56d2-4e27-8838-5caac8f90bcf-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-mwkr7\" (UID: \"b70073f8-56d2-4e27-8838-5caac8f90bcf\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwkr7"
Jul 28 19:37:39.139197 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:39.139180    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdvps\" (UniqueName: \"kubernetes.io/projected/b70073f8-56d2-4e27-8838-5caac8f90bcf-kube-api-access-sdvps\") pod \"cert-manager-cainjector-559cdd8c87-mwkr7\" (UID: \"b70073f8-56d2-4e27-8838-5caac8f90bcf\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwkr7"
Jul 28 19:37:39.239837 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:39.239809    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b70073f8-56d2-4e27-8838-5caac8f90bcf-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-mwkr7\" (UID: \"b70073f8-56d2-4e27-8838-5caac8f90bcf\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwkr7"
Jul 28 19:37:39.239982 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:39.239861    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdvps\" (UniqueName: \"kubernetes.io/projected/b70073f8-56d2-4e27-8838-5caac8f90bcf-kube-api-access-sdvps\") pod \"cert-manager-cainjector-559cdd8c87-mwkr7\" (UID: \"b70073f8-56d2-4e27-8838-5caac8f90bcf\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwkr7"
Jul 28 19:37:39.249429 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:39.249400    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b70073f8-56d2-4e27-8838-5caac8f90bcf-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-mwkr7\" (UID: \"b70073f8-56d2-4e27-8838-5caac8f90bcf\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwkr7"
Jul 28 19:37:39.249590 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:39.249565    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdvps\" (UniqueName: \"kubernetes.io/projected/b70073f8-56d2-4e27-8838-5caac8f90bcf-kube-api-access-sdvps\") pod \"cert-manager-cainjector-559cdd8c87-mwkr7\" (UID: \"b70073f8-56d2-4e27-8838-5caac8f90bcf\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwkr7"
Jul 28 19:37:39.385145 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:39.385119    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwkr7"
Jul 28 19:37:39.503311 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:39.503282    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4" event={"ID":"8143798c-8e3c-4a3b-a14f-af4a15940521","Type":"ContainerStarted","Data":"4cdffa969afd5e57a1a88355c15a18118353d88c163d9b38478ca86e36f023ad"}
Jul 28 19:37:39.514366 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:39.514345    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-mwkr7"]
Jul 28 19:37:39.516732 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:37:39.516706    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb70073f8_56d2_4e27_8838_5caac8f90bcf.slice/crio-8eb035633ee4f585f22b885162e84f2b8cdb5ebdfe7313c3ac304792f68131d9 WatchSource:0}: Error finding container 8eb035633ee4f585f22b885162e84f2b8cdb5ebdfe7313c3ac304792f68131d9: Status 404 returned error can't find the container with id 8eb035633ee4f585f22b885162e84f2b8cdb5ebdfe7313c3ac304792f68131d9
Jul 28 19:37:40.510046 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:40.509957    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwkr7" event={"ID":"b70073f8-56d2-4e27-8838-5caac8f90bcf","Type":"ContainerStarted","Data":"8eb035633ee4f585f22b885162e84f2b8cdb5ebdfe7313c3ac304792f68131d9"}
Jul 28 19:37:40.511941 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:40.511914    2572 generic.go:356] "Generic (PLEG): container finished" podID="8143798c-8e3c-4a3b-a14f-af4a15940521" containerID="4cdffa969afd5e57a1a88355c15a18118353d88c163d9b38478ca86e36f023ad" exitCode=0
Jul 28 19:37:40.512000 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:40.511968    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4" event={"ID":"8143798c-8e3c-4a3b-a14f-af4a15940521","Type":"ContainerDied","Data":"4cdffa969afd5e57a1a88355c15a18118353d88c163d9b38478ca86e36f023ad"}
Jul 28 19:37:40.900174 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:40.900124    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:40.956011 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:40.955984    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:41.518633 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:41.518597    2572 generic.go:356] "Generic (PLEG): container finished" podID="8143798c-8e3c-4a3b-a14f-af4a15940521" containerID="a6288556dc427c01ad2c6b30db4a5b8ae205e4ad1a2e79c8a26d2d779215ab86" exitCode=0
Jul 28 19:37:41.519036 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:41.518670    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4" event={"ID":"8143798c-8e3c-4a3b-a14f-af4a15940521","Type":"ContainerDied","Data":"a6288556dc427c01ad2c6b30db4a5b8ae205e4ad1a2e79c8a26d2d779215ab86"}
Jul 28 19:37:42.648035 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:42.648015    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4"
Jul 28 19:37:42.767727 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:42.767664    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nv8hb\" (UniqueName: \"kubernetes.io/projected/8143798c-8e3c-4a3b-a14f-af4a15940521-kube-api-access-nv8hb\") pod \"8143798c-8e3c-4a3b-a14f-af4a15940521\" (UID: \"8143798c-8e3c-4a3b-a14f-af4a15940521\") "
Jul 28 19:37:42.767727 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:42.767710    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8143798c-8e3c-4a3b-a14f-af4a15940521-util\") pod \"8143798c-8e3c-4a3b-a14f-af4a15940521\" (UID: \"8143798c-8e3c-4a3b-a14f-af4a15940521\") "
Jul 28 19:37:42.767897 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:42.767741    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8143798c-8e3c-4a3b-a14f-af4a15940521-bundle\") pod \"8143798c-8e3c-4a3b-a14f-af4a15940521\" (UID: \"8143798c-8e3c-4a3b-a14f-af4a15940521\") "
Jul 28 19:37:42.768132 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:42.768107    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8143798c-8e3c-4a3b-a14f-af4a15940521-bundle" (OuterVolumeSpecName: "bundle") pod "8143798c-8e3c-4a3b-a14f-af4a15940521" (UID: "8143798c-8e3c-4a3b-a14f-af4a15940521"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:37:42.769622 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:42.769592    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8143798c-8e3c-4a3b-a14f-af4a15940521-kube-api-access-nv8hb" (OuterVolumeSpecName: "kube-api-access-nv8hb") pod "8143798c-8e3c-4a3b-a14f-af4a15940521" (UID: "8143798c-8e3c-4a3b-a14f-af4a15940521"). InnerVolumeSpecName "kube-api-access-nv8hb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:37:42.772154 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:42.772135    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8143798c-8e3c-4a3b-a14f-af4a15940521-util" (OuterVolumeSpecName: "util") pod "8143798c-8e3c-4a3b-a14f-af4a15940521" (UID: "8143798c-8e3c-4a3b-a14f-af4a15940521"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:37:42.868245 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:42.868221    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8143798c-8e3c-4a3b-a14f-af4a15940521-bundle\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:37:42.868245 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:42.868242    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv8hb\" (UniqueName: \"kubernetes.io/projected/8143798c-8e3c-4a3b-a14f-af4a15940521-kube-api-access-nv8hb\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:37:42.868395 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:42.868252    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8143798c-8e3c-4a3b-a14f-af4a15940521-util\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:37:43.277573 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:43.277538    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:43.397747 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:43.397723    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:43.528433 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:43.528370    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4"
Jul 28 19:37:43.528433 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:43.528381    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcffq4" event={"ID":"8143798c-8e3c-4a3b-a14f-af4a15940521","Type":"ContainerDied","Data":"836cf28160e43e7d51961756da4cc1c19f679201ae83b62526210c06203289c2"}
Jul 28 19:37:43.528433 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:43.528426    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="836cf28160e43e7d51961756da4cc1c19f679201ae83b62526210c06203289c2"
Jul 28 19:37:43.530006 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:43.529984    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwkr7" event={"ID":"b70073f8-56d2-4e27-8838-5caac8f90bcf","Type":"ContainerStarted","Data":"416c8adb138e2c4199c48e768271fecb1706a9d076f5fea02c18070f4c945ec3"}
Jul 28 19:37:43.545212 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:43.545173    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwkr7" podStartSLOduration=1.5597517220000001 podStartE2EDuration="4.545161248s" totalImagesPullingTime="2.985409526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:37:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:43.54434347 +0000 UTC m=+489.132006414" watchObservedRunningTime="2026-07-28 19:37:43.545161248 +0000 UTC m=+489.132824191"
Jul 28 19:37:47.072151 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:47.072112    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:47.971152 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:47.971128    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:48.070549 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:48.070523    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:49.688264 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.688232    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d2f56"]
Jul 28 19:37:49.688732 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.688512    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8143798c-8e3c-4a3b-a14f-af4a15940521" containerName="util"
Jul 28 19:37:49.688732 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.688524    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8143798c-8e3c-4a3b-a14f-af4a15940521" containerName="util"
Jul 28 19:37:49.688732 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.688537    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8143798c-8e3c-4a3b-a14f-af4a15940521" containerName="extract"
Jul 28 19:37:49.688732 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.688543    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8143798c-8e3c-4a3b-a14f-af4a15940521" containerName="extract"
Jul 28 19:37:49.688732 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.688550    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8143798c-8e3c-4a3b-a14f-af4a15940521" containerName="pull"
Jul 28 19:37:49.688732 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.688555    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8143798c-8e3c-4a3b-a14f-af4a15940521" containerName="pull"
Jul 28 19:37:49.688732 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.688605    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8143798c-8e3c-4a3b-a14f-af4a15940521" containerName="extract"
Jul 28 19:37:49.691257 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.691240    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d2f56"
Jul 28 19:37:49.696182 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.696153    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-tk54g\""
Jul 28 19:37:49.696548 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.696527    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:37:49.699613 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.699573    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 19:37:49.702396 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.702368    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d2f56"]
Jul 28 19:37:49.816354 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.816331    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnjzd\" (UniqueName: \"kubernetes.io/projected/824928c8-3d59-49b6-adc0-77d1df5841d0-kube-api-access-lnjzd\") pod \"openshift-lws-operator-fd8ccff4c-d2f56\" (UID: \"824928c8-3d59-49b6-adc0-77d1df5841d0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d2f56"
Jul 28 19:37:49.816481 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.816381    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/824928c8-3d59-49b6-adc0-77d1df5841d0-tmp\") pod \"openshift-lws-operator-fd8ccff4c-d2f56\" (UID: \"824928c8-3d59-49b6-adc0-77d1df5841d0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d2f56"
Jul 28 19:37:49.917290 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.917262    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/824928c8-3d59-49b6-adc0-77d1df5841d0-tmp\") pod \"openshift-lws-operator-fd8ccff4c-d2f56\" (UID: \"824928c8-3d59-49b6-adc0-77d1df5841d0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d2f56"
Jul 28 19:37:49.917424 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.917306    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnjzd\" (UniqueName: \"kubernetes.io/projected/824928c8-3d59-49b6-adc0-77d1df5841d0-kube-api-access-lnjzd\") pod \"openshift-lws-operator-fd8ccff4c-d2f56\" (UID: \"824928c8-3d59-49b6-adc0-77d1df5841d0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d2f56"
Jul 28 19:37:49.917695 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.917669    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/824928c8-3d59-49b6-adc0-77d1df5841d0-tmp\") pod \"openshift-lws-operator-fd8ccff4c-d2f56\" (UID: \"824928c8-3d59-49b6-adc0-77d1df5841d0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d2f56"
Jul 28 19:37:49.925353 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:49.925322    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnjzd\" (UniqueName: \"kubernetes.io/projected/824928c8-3d59-49b6-adc0-77d1df5841d0-kube-api-access-lnjzd\") pod \"openshift-lws-operator-fd8ccff4c-d2f56\" (UID: \"824928c8-3d59-49b6-adc0-77d1df5841d0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d2f56"
Jul 28 19:37:50.005695 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:50.005627    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d2f56"
Jul 28 19:37:50.133613 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:50.133497    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d2f56"]
Jul 28 19:37:50.136379 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:37:50.136352    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod824928c8_3d59_49b6_adc0_77d1df5841d0.slice/crio-525594746d67a00e4efaf0ffb22fdc0f151e3f9d4e4a3b614cd76df771397d31 WatchSource:0}: Error finding container 525594746d67a00e4efaf0ffb22fdc0f151e3f9d4e4a3b614cd76df771397d31: Status 404 returned error can't find the container with id 525594746d67a00e4efaf0ffb22fdc0f151e3f9d4e4a3b614cd76df771397d31
Jul 28 19:37:50.560266 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:50.560233    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d2f56" event={"ID":"824928c8-3d59-49b6-adc0-77d1df5841d0","Type":"ContainerStarted","Data":"525594746d67a00e4efaf0ffb22fdc0f151e3f9d4e4a3b614cd76df771397d31"}
Jul 28 19:37:52.571582 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:52.571552    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d2f56" event={"ID":"824928c8-3d59-49b6-adc0-77d1df5841d0","Type":"ContainerStarted","Data":"865e74c4274ba2619486ef8da3422018fe72b8d9db7c62e854ab21e1f3e513cb"}
Jul 28 19:37:52.586507 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:52.586461    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d2f56" podStartSLOduration=1.76782372 podStartE2EDuration="3.586442864s" totalImagesPullingTime="1.818619144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:37:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:52.585779594 +0000 UTC m=+498.173442541" watchObservedRunningTime="2026-07-28 19:37:52.586442864 +0000 UTC m=+498.174105811"
Jul 28 19:37:53.200285 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:53.200254    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:53.301237 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:53.301204    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:54.933516 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:54.933485    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:55.201884 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:55.201814    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4"]
Jul 28 19:37:55.209531 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:55.209512    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4"
Jul 28 19:37:55.212245 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:55.212220    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 19:37:55.212430 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:55.212390    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 19:37:55.213549 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:55.213493    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-whzjq\""
Jul 28 19:37:55.215272 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:55.215250    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4"]
Jul 28 19:37:55.354186 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:55.354155    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mjhk\" (UniqueName: \"kubernetes.io/projected/08e1dd53-d73e-414a-93cd-e931eb0ef58d-kube-api-access-9mjhk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4\" (UID: \"08e1dd53-d73e-414a-93cd-e931eb0ef58d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4"
Jul 28 19:37:55.354284 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:55.354192    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08e1dd53-d73e-414a-93cd-e931eb0ef58d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4\" (UID: \"08e1dd53-d73e-414a-93cd-e931eb0ef58d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4"
Jul 28 19:37:55.354284 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:55.354221    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08e1dd53-d73e-414a-93cd-e931eb0ef58d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4\" (UID: \"08e1dd53-d73e-414a-93cd-e931eb0ef58d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4"
Jul 28 19:37:55.454634 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:55.454574    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mjhk\" (UniqueName: \"kubernetes.io/projected/08e1dd53-d73e-414a-93cd-e931eb0ef58d-kube-api-access-9mjhk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4\" (UID: \"08e1dd53-d73e-414a-93cd-e931eb0ef58d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4"
Jul 28 19:37:55.454634 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:55.454608    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08e1dd53-d73e-414a-93cd-e931eb0ef58d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4\" (UID: \"08e1dd53-d73e-414a-93cd-e931eb0ef58d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4"
Jul 28 19:37:55.454634 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:55.454627    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08e1dd53-d73e-414a-93cd-e931eb0ef58d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4\" (UID: \"08e1dd53-d73e-414a-93cd-e931eb0ef58d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4"
Jul 28 19:37:55.455036 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:55.455020    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08e1dd53-d73e-414a-93cd-e931eb0ef58d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4\" (UID: \"08e1dd53-d73e-414a-93cd-e931eb0ef58d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4"
Jul 28 19:37:55.455110 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:55.455043    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08e1dd53-d73e-414a-93cd-e931eb0ef58d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4\" (UID: \"08e1dd53-d73e-414a-93cd-e931eb0ef58d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4"
Jul 28 19:37:55.462314 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:55.462292    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mjhk\" (UniqueName: \"kubernetes.io/projected/08e1dd53-d73e-414a-93cd-e931eb0ef58d-kube-api-access-9mjhk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4\" (UID: \"08e1dd53-d73e-414a-93cd-e931eb0ef58d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4"
Jul 28 19:37:55.520179 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:55.520159    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4"
Jul 28 19:37:55.645682 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:55.645654    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4"]
Jul 28 19:37:55.648712 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:37:55.648678    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08e1dd53_d73e_414a_93cd_e931eb0ef58d.slice/crio-edaea98cc7ec4601d277ec2f6b6d5d506c619edbf46ae9f55889147e9253f3ae WatchSource:0}: Error finding container edaea98cc7ec4601d277ec2f6b6d5d506c619edbf46ae9f55889147e9253f3ae: Status 404 returned error can't find the container with id edaea98cc7ec4601d277ec2f6b6d5d506c619edbf46ae9f55889147e9253f3ae
Jul 28 19:37:56.590736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:56.590707    2572 generic.go:356] "Generic (PLEG): container finished" podID="08e1dd53-d73e-414a-93cd-e931eb0ef58d" containerID="d85340926905623baab9d1252cd20074819f9dd92a439ef8e3b05768b4428af2" exitCode=0
Jul 28 19:37:56.591044 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:56.590803    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4" event={"ID":"08e1dd53-d73e-414a-93cd-e931eb0ef58d","Type":"ContainerDied","Data":"d85340926905623baab9d1252cd20074819f9dd92a439ef8e3b05768b4428af2"}
Jul 28 19:37:56.591044 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:56.590853    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4" event={"ID":"08e1dd53-d73e-414a-93cd-e931eb0ef58d","Type":"ContainerStarted","Data":"edaea98cc7ec4601d277ec2f6b6d5d506c619edbf46ae9f55889147e9253f3ae"}
Jul 28 19:37:57.597488 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:57.597450    2572 generic.go:356] "Generic (PLEG): container finished" podID="08e1dd53-d73e-414a-93cd-e931eb0ef58d" containerID="c72ca1f78efe4955267336d0b81574c5569770211d4960a70b71ed8556289bf0" exitCode=0
Jul 28 19:37:57.597886 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:57.597537    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4" event={"ID":"08e1dd53-d73e-414a-93cd-e931eb0ef58d","Type":"ContainerDied","Data":"c72ca1f78efe4955267336d0b81574c5569770211d4960a70b71ed8556289bf0"}
Jul 28 19:37:58.605717 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:58.605678    2572 generic.go:356] "Generic (PLEG): container finished" podID="08e1dd53-d73e-414a-93cd-e931eb0ef58d" containerID="7412026badc01c6e90a270f513ddcf485acdc66489408e9574422f6181df5a2a" exitCode=0
Jul 28 19:37:58.606061 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:58.605756    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4" event={"ID":"08e1dd53-d73e-414a-93cd-e931eb0ef58d","Type":"ContainerDied","Data":"7412026badc01c6e90a270f513ddcf485acdc66489408e9574422f6181df5a2a"}
Jul 28 19:37:59.746567 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:59.746545    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4"
Jul 28 19:37:59.886185 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:59.886129    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08e1dd53-d73e-414a-93cd-e931eb0ef58d-util\") pod \"08e1dd53-d73e-414a-93cd-e931eb0ef58d\" (UID: \"08e1dd53-d73e-414a-93cd-e931eb0ef58d\") "
Jul 28 19:37:59.886185 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:59.886162    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mjhk\" (UniqueName: \"kubernetes.io/projected/08e1dd53-d73e-414a-93cd-e931eb0ef58d-kube-api-access-9mjhk\") pod \"08e1dd53-d73e-414a-93cd-e931eb0ef58d\" (UID: \"08e1dd53-d73e-414a-93cd-e931eb0ef58d\") "
Jul 28 19:37:59.886377 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:59.886198    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08e1dd53-d73e-414a-93cd-e931eb0ef58d-bundle\") pod \"08e1dd53-d73e-414a-93cd-e931eb0ef58d\" (UID: \"08e1dd53-d73e-414a-93cd-e931eb0ef58d\") "
Jul 28 19:37:59.886902 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:59.886871    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08e1dd53-d73e-414a-93cd-e931eb0ef58d-bundle" (OuterVolumeSpecName: "bundle") pod "08e1dd53-d73e-414a-93cd-e931eb0ef58d" (UID: "08e1dd53-d73e-414a-93cd-e931eb0ef58d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:37:59.888154 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:59.888131    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08e1dd53-d73e-414a-93cd-e931eb0ef58d-kube-api-access-9mjhk" (OuterVolumeSpecName: "kube-api-access-9mjhk") pod "08e1dd53-d73e-414a-93cd-e931eb0ef58d" (UID: "08e1dd53-d73e-414a-93cd-e931eb0ef58d"). InnerVolumeSpecName "kube-api-access-9mjhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:37:59.891424 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:59.891399    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08e1dd53-d73e-414a-93cd-e931eb0ef58d-util" (OuterVolumeSpecName: "util") pod "08e1dd53-d73e-414a-93cd-e931eb0ef58d" (UID: "08e1dd53-d73e-414a-93cd-e931eb0ef58d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:37:59.987171 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:59.987149    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08e1dd53-d73e-414a-93cd-e931eb0ef58d-util\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:37:59.987171 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:59.987171    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mjhk\" (UniqueName: \"kubernetes.io/projected/08e1dd53-d73e-414a-93cd-e931eb0ef58d-kube-api-access-9mjhk\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:37:59.987297 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:37:59.987181    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08e1dd53-d73e-414a-93cd-e931eb0ef58d-bundle\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:38:00.616973 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:00.616950    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4"
Jul 28 19:38:00.617124 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:00.616945    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9qjw4" event={"ID":"08e1dd53-d73e-414a-93cd-e931eb0ef58d","Type":"ContainerDied","Data":"edaea98cc7ec4601d277ec2f6b6d5d506c619edbf46ae9f55889147e9253f3ae"}
Jul 28 19:38:00.617124 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:00.617085    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="edaea98cc7ec4601d277ec2f6b6d5d506c619edbf46ae9f55889147e9253f3ae"
Jul 28 19:38:10.949006 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:10.948962    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:11.047891 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.047864    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:11.385754 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.385726    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:11.605846 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.605816    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:11.711271 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.711197    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:11.737053 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.737025    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"]
Jul 28 19:38:11.737356 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.737341    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08e1dd53-d73e-414a-93cd-e931eb0ef58d" containerName="pull"
Jul 28 19:38:11.737422 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.737357    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="08e1dd53-d73e-414a-93cd-e931eb0ef58d" containerName="pull"
Jul 28 19:38:11.737422 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.737374    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08e1dd53-d73e-414a-93cd-e931eb0ef58d" containerName="extract"
Jul 28 19:38:11.737422 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.737383    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="08e1dd53-d73e-414a-93cd-e931eb0ef58d" containerName="extract"
Jul 28 19:38:11.737422 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.737393    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08e1dd53-d73e-414a-93cd-e931eb0ef58d" containerName="util"
Jul 28 19:38:11.737422 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.737398    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="08e1dd53-d73e-414a-93cd-e931eb0ef58d" containerName="util"
Jul 28 19:38:11.737573 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.737477    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="08e1dd53-d73e-414a-93cd-e931eb0ef58d" containerName="extract"
Jul 28 19:38:11.742013 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.741993    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:11.745510 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.745480    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 19:38:11.745657 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.745640    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 19:38:11.745739 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.745708    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 19:38:11.745798 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.745741    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gvzp5\""
Jul 28 19:38:11.745853 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.745837    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 19:38:11.754450 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.754428    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"]
Jul 28 19:38:11.862778 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.862754    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8ksg\" (UniqueName: \"kubernetes.io/projected/4b22cfe6-866f-4639-9caf-18ce24cf5748-kube-api-access-v8ksg\") pod \"opendatahub-operator-controller-manager-9ddd48bbd-7tqxm\" (UID: \"4b22cfe6-866f-4639-9caf-18ce24cf5748\") " pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:11.862899 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.862805    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4b22cfe6-866f-4639-9caf-18ce24cf5748-manifests\") pod \"opendatahub-operator-controller-manager-9ddd48bbd-7tqxm\" (UID: \"4b22cfe6-866f-4639-9caf-18ce24cf5748\") " pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:11.862899 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.862836    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b22cfe6-866f-4639-9caf-18ce24cf5748-tmp\") pod \"opendatahub-operator-controller-manager-9ddd48bbd-7tqxm\" (UID: \"4b22cfe6-866f-4639-9caf-18ce24cf5748\") " pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:11.862899 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.862866    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4b22cfe6-866f-4639-9caf-18ce24cf5748-webhook-cert\") pod \"opendatahub-operator-controller-manager-9ddd48bbd-7tqxm\" (UID: \"4b22cfe6-866f-4639-9caf-18ce24cf5748\") " pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:11.863022 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.862897    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4b22cfe6-866f-4639-9caf-18ce24cf5748-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9ddd48bbd-7tqxm\" (UID: \"4b22cfe6-866f-4639-9caf-18ce24cf5748\") " pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:11.883464 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.883437    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:11.963503 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.963449    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4b22cfe6-866f-4639-9caf-18ce24cf5748-manifests\") pod \"opendatahub-operator-controller-manager-9ddd48bbd-7tqxm\" (UID: \"4b22cfe6-866f-4639-9caf-18ce24cf5748\") " pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:11.963503 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.963495    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b22cfe6-866f-4639-9caf-18ce24cf5748-tmp\") pod \"opendatahub-operator-controller-manager-9ddd48bbd-7tqxm\" (UID: \"4b22cfe6-866f-4639-9caf-18ce24cf5748\") " pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:11.963918 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.963551    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4b22cfe6-866f-4639-9caf-18ce24cf5748-webhook-cert\") pod \"opendatahub-operator-controller-manager-9ddd48bbd-7tqxm\" (UID: \"4b22cfe6-866f-4639-9caf-18ce24cf5748\") " pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:11.963918 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.963587    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4b22cfe6-866f-4639-9caf-18ce24cf5748-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9ddd48bbd-7tqxm\" (UID: \"4b22cfe6-866f-4639-9caf-18ce24cf5748\") " pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:11.963918 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.963868    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8ksg\" (UniqueName: \"kubernetes.io/projected/4b22cfe6-866f-4639-9caf-18ce24cf5748-kube-api-access-v8ksg\") pod \"opendatahub-operator-controller-manager-9ddd48bbd-7tqxm\" (UID: \"4b22cfe6-866f-4639-9caf-18ce24cf5748\") " pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:11.963918 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.963871    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4b22cfe6-866f-4639-9caf-18ce24cf5748-manifests\") pod \"opendatahub-operator-controller-manager-9ddd48bbd-7tqxm\" (UID: \"4b22cfe6-866f-4639-9caf-18ce24cf5748\") " pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:11.965793 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.965770    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b22cfe6-866f-4639-9caf-18ce24cf5748-tmp\") pod \"opendatahub-operator-controller-manager-9ddd48bbd-7tqxm\" (UID: \"4b22cfe6-866f-4639-9caf-18ce24cf5748\") " pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:11.966111 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.966091    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4b22cfe6-866f-4639-9caf-18ce24cf5748-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9ddd48bbd-7tqxm\" (UID: \"4b22cfe6-866f-4639-9caf-18ce24cf5748\") " pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:11.966162 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.966096    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4b22cfe6-866f-4639-9caf-18ce24cf5748-webhook-cert\") pod \"opendatahub-operator-controller-manager-9ddd48bbd-7tqxm\" (UID: \"4b22cfe6-866f-4639-9caf-18ce24cf5748\") " pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:11.972602 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.972581    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8ksg\" (UniqueName: \"kubernetes.io/projected/4b22cfe6-866f-4639-9caf-18ce24cf5748-kube-api-access-v8ksg\") pod \"opendatahub-operator-controller-manager-9ddd48bbd-7tqxm\" (UID: \"4b22cfe6-866f-4639-9caf-18ce24cf5748\") " pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:11.993356 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:11.993335    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:12.053001 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:12.052979    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:12.094782 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:12.094758    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:12.210931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:12.210894    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:12.211604 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:12.211575    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"]
Jul 28 19:38:12.214262 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:38:12.214204    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b22cfe6_866f_4639_9caf_18ce24cf5748.slice/crio-dba578f7fd56c0a16312001a299df8a1b9406681dac4836243df74b857429d5a WatchSource:0}: Error finding container dba578f7fd56c0a16312001a299df8a1b9406681dac4836243df74b857429d5a: Status 404 returned error can't find the container with id dba578f7fd56c0a16312001a299df8a1b9406681dac4836243df74b857429d5a
Jul 28 19:38:12.301125 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:12.301057    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:12.395336 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:12.395310    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:12.668866 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:12.668842    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm" event={"ID":"4b22cfe6-866f-4639-9caf-18ce24cf5748","Type":"ContainerStarted","Data":"dba578f7fd56c0a16312001a299df8a1b9406681dac4836243df74b857429d5a"}
Jul 28 19:38:15.685862 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:15.685836    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm" event={"ID":"4b22cfe6-866f-4639-9caf-18ce24cf5748","Type":"ContainerStarted","Data":"c779d048ef8845ca0c95c2d5f6c7fc4dcb25572e411684be36a42d9e9424aeee"}
Jul 28 19:38:16.691853 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:16.691783    2572 generic.go:356] "Generic (PLEG): container finished" podID="4b22cfe6-866f-4639-9caf-18ce24cf5748" containerID="c779d048ef8845ca0c95c2d5f6c7fc4dcb25572e411684be36a42d9e9424aeee" exitCode=0
Jul 28 19:38:16.692201 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:16.691871    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm" event={"ID":"4b22cfe6-866f-4639-9caf-18ce24cf5748","Type":"ContainerDied","Data":"c779d048ef8845ca0c95c2d5f6c7fc4dcb25572e411684be36a42d9e9424aeee"}
Jul 28 19:38:17.699550 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:17.699505    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm" event={"ID":"4b22cfe6-866f-4639-9caf-18ce24cf5748","Type":"ContainerStarted","Data":"49ce0102fb37ec783f1438a065833443e6769bfa7b0eed6f9664ca95b8c8a8a4"}
Jul 28 19:38:17.699939 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:17.699596    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:17.721523 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:17.721466    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm" podStartSLOduration=2.334389167 podStartE2EDuration="6.721452123s" totalImagesPullingTime="3.344996014s" totalInitContainerRuntime="1.042066942s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:38:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:38:17.718479723 +0000 UTC m=+523.306142686" watchObservedRunningTime="2026-07-28 19:38:17.721452123 +0000 UTC m=+523.309115068"
Jul 28 19:38:21.189326 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:21.189294    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:21.289685 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:21.289652    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:24.966763 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:24.966727    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:28.706306 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:28.706232    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9ddd48bbd-7tqxm"
Jul 28 19:38:30.774338 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:30.774303    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:30.911262 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:30.911232    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:31.304574 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.304541    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:31.425355 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.425328    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-ccfbc6b9-fw9kp"]
Jul 28 19:38:31.428645 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.428630    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-ccfbc6b9-fw9kp"
Jul 28 19:38:31.431019 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.430999    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 19:38:31.431165 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.431127    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 19:38:31.431165 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.431161    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 19:38:31.431386 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.431366    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-c8g9l\""
Jul 28 19:38:31.431386 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.431384    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 19:38:31.442750 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.442731    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-ccfbc6b9-fw9kp"]
Jul 28 19:38:31.505415 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.505395    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5png6\" (UniqueName: \"kubernetes.io/projected/a29780d1-d9e7-4b75-8b27-a34d95aacec2-kube-api-access-5png6\") pod \"kube-auth-proxy-ccfbc6b9-fw9kp\" (UID: \"a29780d1-d9e7-4b75-8b27-a34d95aacec2\") " pod="openshift-ingress/kube-auth-proxy-ccfbc6b9-fw9kp"
Jul 28 19:38:31.505513 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.505426    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a29780d1-d9e7-4b75-8b27-a34d95aacec2-tmp\") pod \"kube-auth-proxy-ccfbc6b9-fw9kp\" (UID: \"a29780d1-d9e7-4b75-8b27-a34d95aacec2\") " pod="openshift-ingress/kube-auth-proxy-ccfbc6b9-fw9kp"
Jul 28 19:38:31.505513 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.505452    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a29780d1-d9e7-4b75-8b27-a34d95aacec2-tls-certs\") pod \"kube-auth-proxy-ccfbc6b9-fw9kp\" (UID: \"a29780d1-d9e7-4b75-8b27-a34d95aacec2\") " pod="openshift-ingress/kube-auth-proxy-ccfbc6b9-fw9kp"
Jul 28 19:38:31.603617 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.603596    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-zxvjp"]
Jul 28 19:38:31.605716 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.605695    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a29780d1-d9e7-4b75-8b27-a34d95aacec2-tmp\") pod \"kube-auth-proxy-ccfbc6b9-fw9kp\" (UID: \"a29780d1-d9e7-4b75-8b27-a34d95aacec2\") " pod="openshift-ingress/kube-auth-proxy-ccfbc6b9-fw9kp"
Jul 28 19:38:31.605797 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.605731    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a29780d1-d9e7-4b75-8b27-a34d95aacec2-tls-certs\") pod \"kube-auth-proxy-ccfbc6b9-fw9kp\" (UID: \"a29780d1-d9e7-4b75-8b27-a34d95aacec2\") " pod="openshift-ingress/kube-auth-proxy-ccfbc6b9-fw9kp"
Jul 28 19:38:31.605797 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.605768    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5png6\" (UniqueName: \"kubernetes.io/projected/a29780d1-d9e7-4b75-8b27-a34d95aacec2-kube-api-access-5png6\") pod \"kube-auth-proxy-ccfbc6b9-fw9kp\" (UID: \"a29780d1-d9e7-4b75-8b27-a34d95aacec2\") " pod="openshift-ingress/kube-auth-proxy-ccfbc6b9-fw9kp"
Jul 28 19:38:31.605864 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:38:31.605839    2572 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 28 19:38:31.605905 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:38:31.605896    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a29780d1-d9e7-4b75-8b27-a34d95aacec2-tls-certs podName:a29780d1-d9e7-4b75-8b27-a34d95aacec2 nodeName:}" failed. No retries permitted until 2026-07-28 19:38:32.105881888 +0000 UTC m=+537.693544814 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a29780d1-d9e7-4b75-8b27-a34d95aacec2-tls-certs") pod "kube-auth-proxy-ccfbc6b9-fw9kp" (UID: "a29780d1-d9e7-4b75-8b27-a34d95aacec2") : secret "kube-auth-proxy-tls" not found
Jul 28 19:38:31.606727 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.606711    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-zxvjp"
Jul 28 19:38:31.607986 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.607968    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a29780d1-d9e7-4b75-8b27-a34d95aacec2-tmp\") pod \"kube-auth-proxy-ccfbc6b9-fw9kp\" (UID: \"a29780d1-d9e7-4b75-8b27-a34d95aacec2\") " pod="openshift-ingress/kube-auth-proxy-ccfbc6b9-fw9kp"
Jul 28 19:38:31.611285 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.611262    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 28 19:38:31.612400 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.612379    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-sm476\""
Jul 28 19:38:31.618643 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.618617    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-zxvjp"]
Jul 28 19:38:31.622019 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.621999    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5png6\" (UniqueName: \"kubernetes.io/projected/a29780d1-d9e7-4b75-8b27-a34d95aacec2-kube-api-access-5png6\") pod \"kube-auth-proxy-ccfbc6b9-fw9kp\" (UID: \"a29780d1-d9e7-4b75-8b27-a34d95aacec2\") " pod="openshift-ingress/kube-auth-proxy-ccfbc6b9-fw9kp"
Jul 28 19:38:31.648687 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.648665    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:31.706106 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.706083    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00096bf5-8c3d-4c1e-9860-ccdb60b4b795-cert\") pod \"odh-model-controller-7998c67dc4-zxvjp\" (UID: \"00096bf5-8c3d-4c1e-9860-ccdb60b4b795\") " pod="opendatahub/odh-model-controller-7998c67dc4-zxvjp"
Jul 28 19:38:31.706220 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.706122    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hng57\" (UniqueName: \"kubernetes.io/projected/00096bf5-8c3d-4c1e-9860-ccdb60b4b795-kube-api-access-hng57\") pod \"odh-model-controller-7998c67dc4-zxvjp\" (UID: \"00096bf5-8c3d-4c1e-9860-ccdb60b4b795\") " pod="opendatahub/odh-model-controller-7998c67dc4-zxvjp"
Jul 28 19:38:31.741554 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.741528    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:31.807402 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.807380    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00096bf5-8c3d-4c1e-9860-ccdb60b4b795-cert\") pod \"odh-model-controller-7998c67dc4-zxvjp\" (UID: \"00096bf5-8c3d-4c1e-9860-ccdb60b4b795\") " pod="opendatahub/odh-model-controller-7998c67dc4-zxvjp"
Jul 28 19:38:31.807715 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.807415    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hng57\" (UniqueName: \"kubernetes.io/projected/00096bf5-8c3d-4c1e-9860-ccdb60b4b795-kube-api-access-hng57\") pod \"odh-model-controller-7998c67dc4-zxvjp\" (UID: \"00096bf5-8c3d-4c1e-9860-ccdb60b4b795\") " pod="opendatahub/odh-model-controller-7998c67dc4-zxvjp"
Jul 28 19:38:31.809568 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.809552    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00096bf5-8c3d-4c1e-9860-ccdb60b4b795-cert\") pod \"odh-model-controller-7998c67dc4-zxvjp\" (UID: \"00096bf5-8c3d-4c1e-9860-ccdb60b4b795\") " pod="opendatahub/odh-model-controller-7998c67dc4-zxvjp"
Jul 28 19:38:31.819155 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.819137    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hng57\" (UniqueName: \"kubernetes.io/projected/00096bf5-8c3d-4c1e-9860-ccdb60b4b795-kube-api-access-hng57\") pod \"odh-model-controller-7998c67dc4-zxvjp\" (UID: \"00096bf5-8c3d-4c1e-9860-ccdb60b4b795\") " pod="opendatahub/odh-model-controller-7998c67dc4-zxvjp"
Jul 28 19:38:31.842979 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.842957    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:31.932272 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.932217    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-zxvjp"
Jul 28 19:38:31.956832 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:31.956811    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:32.083018 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.082997    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-zxvjp"]
Jul 28 19:38:32.085355 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:38:32.085326    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00096bf5_8c3d_4c1e_9860_ccdb60b4b795.slice/crio-aeab7965272b6e840a3aef4b1fcef5b6da4cca31938f3db09fe0f76d8f750c63 WatchSource:0}: Error finding container aeab7965272b6e840a3aef4b1fcef5b6da4cca31938f3db09fe0f76d8f750c63: Status 404 returned error can't find the container with id aeab7965272b6e840a3aef4b1fcef5b6da4cca31938f3db09fe0f76d8f750c63
Jul 28 19:38:32.110056 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.110028    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a29780d1-d9e7-4b75-8b27-a34d95aacec2-tls-certs\") pod \"kube-auth-proxy-ccfbc6b9-fw9kp\" (UID: \"a29780d1-d9e7-4b75-8b27-a34d95aacec2\") " pod="openshift-ingress/kube-auth-proxy-ccfbc6b9-fw9kp"
Jul 28 19:38:32.112104 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.112087    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a29780d1-d9e7-4b75-8b27-a34d95aacec2-tls-certs\") pod \"kube-auth-proxy-ccfbc6b9-fw9kp\" (UID: \"a29780d1-d9e7-4b75-8b27-a34d95aacec2\") " pod="openshift-ingress/kube-auth-proxy-ccfbc6b9-fw9kp"
Jul 28 19:38:32.339954 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.339929    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-ccfbc6b9-fw9kp"
Jul 28 19:38:32.350029 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.350007    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:32.417107 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.417056    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fnmjf"]
Jul 28 19:38:32.420672 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.420645    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fnmjf"
Jul 28 19:38:32.423920 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.423898    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 28 19:38:32.424043 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.423898    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-pjtgm\""
Jul 28 19:38:32.442126 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.442101    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fnmjf"]
Jul 28 19:38:32.477428 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.477372    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-ccfbc6b9-fw9kp"]
Jul 28 19:38:32.479362 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:38:32.479338    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda29780d1_d9e7_4b75_8b27_a34d95aacec2.slice/crio-22611a3a1d4cdba56d78b45217244b2486e0ea7353a2a750f29cefe7c7c76fa8 WatchSource:0}: Error finding container 22611a3a1d4cdba56d78b45217244b2486e0ea7353a2a750f29cefe7c7c76fa8: Status 404 returned error can't find the container with id 22611a3a1d4cdba56d78b45217244b2486e0ea7353a2a750f29cefe7c7c76fa8
Jul 28 19:38:32.511966 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.511944    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:32.512434 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.512419    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07192589-3f2b-4d02-aee0-35087fd22657-tls-certs\") pod \"model-serving-api-54d89bd5fc-fnmjf\" (UID: \"07192589-3f2b-4d02-aee0-35087fd22657\") " pod="opendatahub/model-serving-api-54d89bd5fc-fnmjf"
Jul 28 19:38:32.512497 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.512445    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj49n\" (UniqueName: \"kubernetes.io/projected/07192589-3f2b-4d02-aee0-35087fd22657-kube-api-access-vj49n\") pod \"model-serving-api-54d89bd5fc-fnmjf\" (UID: \"07192589-3f2b-4d02-aee0-35087fd22657\") " pod="opendatahub/model-serving-api-54d89bd5fc-fnmjf"
Jul 28 19:38:32.612873 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.612826    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj49n\" (UniqueName: \"kubernetes.io/projected/07192589-3f2b-4d02-aee0-35087fd22657-kube-api-access-vj49n\") pod \"model-serving-api-54d89bd5fc-fnmjf\" (UID: \"07192589-3f2b-4d02-aee0-35087fd22657\") " pod="opendatahub/model-serving-api-54d89bd5fc-fnmjf"
Jul 28 19:38:32.612958 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.612887    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07192589-3f2b-4d02-aee0-35087fd22657-tls-certs\") pod \"model-serving-api-54d89bd5fc-fnmjf\" (UID: \"07192589-3f2b-4d02-aee0-35087fd22657\") " pod="opendatahub/model-serving-api-54d89bd5fc-fnmjf"
Jul 28 19:38:32.615028 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.615013    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07192589-3f2b-4d02-aee0-35087fd22657-tls-certs\") pod \"model-serving-api-54d89bd5fc-fnmjf\" (UID: \"07192589-3f2b-4d02-aee0-35087fd22657\") " pod="opendatahub/model-serving-api-54d89bd5fc-fnmjf"
Jul 28 19:38:32.623988 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.623969    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj49n\" (UniqueName: \"kubernetes.io/projected/07192589-3f2b-4d02-aee0-35087fd22657-kube-api-access-vj49n\") pod \"model-serving-api-54d89bd5fc-fnmjf\" (UID: \"07192589-3f2b-4d02-aee0-35087fd22657\") " pod="opendatahub/model-serving-api-54d89bd5fc-fnmjf"
Jul 28 19:38:32.685706 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.685678    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:32.732456 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.732431    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fnmjf"
Jul 28 19:38:32.750208 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.750057    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:32.781954 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.781907    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-ccfbc6b9-fw9kp" event={"ID":"a29780d1-d9e7-4b75-8b27-a34d95aacec2","Type":"ContainerStarted","Data":"22611a3a1d4cdba56d78b45217244b2486e0ea7353a2a750f29cefe7c7c76fa8"}
Jul 28 19:38:32.784103 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.784062    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-zxvjp" event={"ID":"00096bf5-8c3d-4c1e-9860-ccdb60b4b795","Type":"ContainerStarted","Data":"aeab7965272b6e840a3aef4b1fcef5b6da4cca31938f3db09fe0f76d8f750c63"}
Jul 28 19:38:32.888148 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:32.888120    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fnmjf"]
Jul 28 19:38:32.890707 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:38:32.890675    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07192589_3f2b_4d02_aee0_35087fd22657.slice/crio-7b13d09fc69c27a7736a35e21529eb729708603ffd6537a33b65cb2b94cea505 WatchSource:0}: Error finding container 7b13d09fc69c27a7736a35e21529eb729708603ffd6537a33b65cb2b94cea505: Status 404 returned error can't find the container with id 7b13d09fc69c27a7736a35e21529eb729708603ffd6537a33b65cb2b94cea505
Jul 28 19:38:33.592588 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:33.592552    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:33.791495 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:33.791447    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fnmjf" event={"ID":"07192589-3f2b-4d02-aee0-35087fd22657","Type":"ContainerStarted","Data":"7b13d09fc69c27a7736a35e21529eb729708603ffd6537a33b65cb2b94cea505"}
Jul 28 19:38:35.362306 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:35.362284    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 19:38:35.806129 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:35.806081    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-zxvjp" event={"ID":"00096bf5-8c3d-4c1e-9860-ccdb60b4b795","Type":"ContainerStarted","Data":"df960fbe94df9e547bb810669646e5efa854bbabf83fcbf3b390bee31abe2230"}
Jul 28 19:38:35.806308 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:35.806251    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-zxvjp"
Jul 28 19:38:35.807727 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:35.807701    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-ccfbc6b9-fw9kp" event={"ID":"a29780d1-d9e7-4b75-8b27-a34d95aacec2","Type":"ContainerStarted","Data":"bcf706635f134557c24f01d82d2c6b10460e8b3fbf326f7cb0cf361edf262b3f"}
Jul 28 19:38:35.823526 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:35.823480    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-zxvjp" podStartSLOduration=1.5522742040000002 podStartE2EDuration="4.823459708s" totalImagesPullingTime="3.271185504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:38:35.821100383 +0000 UTC m=+541.408763327" watchObservedRunningTime="2026-07-28 19:38:35.823459708 +0000 UTC m=+541.411122653"
Jul 28 19:38:35.835790 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:35.835747    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-ccfbc6b9-fw9kp" podStartSLOduration=1.9571815639999999 podStartE2EDuration="4.835730468s" totalImagesPullingTime="2.878548904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:38:35.835386383 +0000 UTC m=+541.423049326" watchObservedRunningTime="2026-07-28 19:38:35.835730468 +0000 UTC m=+541.423393414"
Jul 28 19:38:36.813920 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:36.813887    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fnmjf" event={"ID":"07192589-3f2b-4d02-aee0-35087fd22657","Type":"ContainerStarted","Data":"4814feb3543b66a8a13a694708565da66a70c8fe6ecaee633e6e3c10b53325e8"}
Jul 28 19:38:36.814355 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:36.814055    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-fnmjf"
Jul 28 19:38:36.829522 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:36.829483    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-fnmjf" podStartSLOduration=1.599889369 podStartE2EDuration="4.829470486s" totalImagesPullingTime="3.229581117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:38:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:38:36.828833963 +0000 UTC m=+542.416496919" watchObservedRunningTime="2026-07-28 19:38:36.829470486 +0000 UTC m=+542.417133433"
Jul 28 19:38:37.051163 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.051139    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:37.151882 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.151815    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:37.733477 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.733453    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"]
Jul 28 19:38:37.736769 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.736749    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.739377 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.739354    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 19:38:37.739640 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.739621    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-jhcsn\""
Jul 28 19:38:37.750526 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.750505    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"]
Jul 28 19:38:37.753652 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.753632    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.753747 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.753670    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.753747 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.753694    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.753828 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.753748    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46sjk\" (UniqueName: \"kubernetes.io/projected/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-kube-api-access-46sjk\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.753828 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.753808    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.753892 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.753860    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.753924 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.753892    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.753957 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.753934    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.753989 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.753971    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.759596 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.759575    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:37.854567 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.854542    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.855006 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.854574    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.855006 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.854597    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.855006 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.854613    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.855006 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.854630    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46sjk\" (UniqueName: \"kubernetes.io/projected/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-kube-api-access-46sjk\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.855006 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.854662    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.855006 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.854735    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.855006 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.854780    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.855006 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.854822    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.855006 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.854998    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.855533 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.855143    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.855533 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.855209    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.855533 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.855390    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.855533 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.855499    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.857286 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.857261    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.857375 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.857297    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.863330 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.863309    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46sjk\" (UniqueName: \"kubernetes.io/projected/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-kube-api-access-46sjk\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:37.863415 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:37.863310    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef85412a-fbbd-4ff4-8e42-21bc4e30e444-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz\" (UID: \"ef85412a-fbbd-4ff4-8e42-21bc4e30e444\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:38.047096 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.046986    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:38.051395 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.051279    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:38.063887 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.063863    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"]
Jul 28 19:38:38.067518 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.067503    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"
Jul 28 19:38:38.070174 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.070128    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hds79\""
Jul 28 19:38:38.070174 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.070158    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 19:38:38.070444 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.070180    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 19:38:38.070444 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.070233    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 19:38:38.077028 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.076935    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"]
Jul 28 19:38:38.145538 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.145481    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:38.156867 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.156846    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpxwm\" (UniqueName: \"kubernetes.io/projected/b406707d-fe20-4134-afd9-f93560ede974-kube-api-access-mpxwm\") pod \"lws-controller-manager-6c85c6bd85-lqtpg\" (UID: \"b406707d-fe20-4134-afd9-f93560ede974\") " pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"
Jul 28 19:38:38.156975 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.156876    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b406707d-fe20-4134-afd9-f93560ede974-metrics-cert\") pod \"lws-controller-manager-6c85c6bd85-lqtpg\" (UID: \"b406707d-fe20-4134-afd9-f93560ede974\") " pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"
Jul 28 19:38:38.156975 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.156906    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b406707d-fe20-4134-afd9-f93560ede974-manager-config\") pod \"lws-controller-manager-6c85c6bd85-lqtpg\" (UID: \"b406707d-fe20-4134-afd9-f93560ede974\") " pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"
Jul 28 19:38:38.157052 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.157032    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b406707d-fe20-4134-afd9-f93560ede974-cert\") pod \"lws-controller-manager-6c85c6bd85-lqtpg\" (UID: \"b406707d-fe20-4134-afd9-f93560ede974\") " pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"
Jul 28 19:38:38.179900 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.179876    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"]
Jul 28 19:38:38.181838 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:38:38.181810    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef85412a_fbbd_4ff4_8e42_21bc4e30e444.slice/crio-0360d2d8f9de09812a96cbf9a2dcc545cfca6ce67a15c7d9b3420dd1325dc28b WatchSource:0}: Error finding container 0360d2d8f9de09812a96cbf9a2dcc545cfca6ce67a15c7d9b3420dd1325dc28b: Status 404 returned error can't find the container with id 0360d2d8f9de09812a96cbf9a2dcc545cfca6ce67a15c7d9b3420dd1325dc28b
Jul 28 19:38:38.257416 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.257394    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b406707d-fe20-4134-afd9-f93560ede974-cert\") pod \"lws-controller-manager-6c85c6bd85-lqtpg\" (UID: \"b406707d-fe20-4134-afd9-f93560ede974\") " pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"
Jul 28 19:38:38.257513 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.257434    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpxwm\" (UniqueName: \"kubernetes.io/projected/b406707d-fe20-4134-afd9-f93560ede974-kube-api-access-mpxwm\") pod \"lws-controller-manager-6c85c6bd85-lqtpg\" (UID: \"b406707d-fe20-4134-afd9-f93560ede974\") " pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"
Jul 28 19:38:38.257513 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.257455    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b406707d-fe20-4134-afd9-f93560ede974-metrics-cert\") pod \"lws-controller-manager-6c85c6bd85-lqtpg\" (UID: \"b406707d-fe20-4134-afd9-f93560ede974\") " pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"
Jul 28 19:38:38.257513 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.257471    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b406707d-fe20-4134-afd9-f93560ede974-manager-config\") pod \"lws-controller-manager-6c85c6bd85-lqtpg\" (UID: \"b406707d-fe20-4134-afd9-f93560ede974\") " pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"
Jul 28 19:38:38.258048 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.258032    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b406707d-fe20-4134-afd9-f93560ede974-manager-config\") pod \"lws-controller-manager-6c85c6bd85-lqtpg\" (UID: \"b406707d-fe20-4134-afd9-f93560ede974\") " pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"
Jul 28 19:38:38.259628 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.259606    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b406707d-fe20-4134-afd9-f93560ede974-cert\") pod \"lws-controller-manager-6c85c6bd85-lqtpg\" (UID: \"b406707d-fe20-4134-afd9-f93560ede974\") " pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"
Jul 28 19:38:38.259755 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.259676    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b406707d-fe20-4134-afd9-f93560ede974-metrics-cert\") pod \"lws-controller-manager-6c85c6bd85-lqtpg\" (UID: \"b406707d-fe20-4134-afd9-f93560ede974\") " pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"
Jul 28 19:38:38.267240 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.267220    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpxwm\" (UniqueName: \"kubernetes.io/projected/b406707d-fe20-4134-afd9-f93560ede974-kube-api-access-mpxwm\") pod \"lws-controller-manager-6c85c6bd85-lqtpg\" (UID: \"b406707d-fe20-4134-afd9-f93560ede974\") " pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"
Jul 28 19:38:38.379786 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.379762    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"
Jul 28 19:38:38.515594 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.515570    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"]
Jul 28 19:38:38.517364 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:38:38.517339    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb406707d_fe20_4134_afd9_f93560ede974.slice/crio-8e2f951a40012ef125265714a0af5c0b9fe4bec06f05819007ed34313b40926c WatchSource:0}: Error finding container 8e2f951a40012ef125265714a0af5c0b9fe4bec06f05819007ed34313b40926c: Status 404 returned error can't find the container with id 8e2f951a40012ef125265714a0af5c0b9fe4bec06f05819007ed34313b40926c
Jul 28 19:38:38.823891 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.823857    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz" event={"ID":"ef85412a-fbbd-4ff4-8e42-21bc4e30e444","Type":"ContainerStarted","Data":"0360d2d8f9de09812a96cbf9a2dcc545cfca6ce67a15c7d9b3420dd1325dc28b"}
Jul 28 19:38:38.824964 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:38.824940    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg" event={"ID":"b406707d-fe20-4134-afd9-f93560ede974","Type":"ContainerStarted","Data":"8e2f951a40012ef125265714a0af5c0b9fe4bec06f05819007ed34313b40926c"}
Jul 28 19:38:41.338548 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:41.338512    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:38:41.338812 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:41.338600    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:38:41.338812 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:41.338639    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:38:41.843437 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:41.843398    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz" event={"ID":"ef85412a-fbbd-4ff4-8e42-21bc4e30e444","Type":"ContainerStarted","Data":"9112d9d7914c38415aaa3841b43c5c40172c37a08cd345aae0c8382c96122027"}
Jul 28 19:38:41.845679 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:41.845640    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg" event={"ID":"b406707d-fe20-4134-afd9-f93560ede974","Type":"ContainerStarted","Data":"abb585c2d6ef5662a3f5865a8f65244b16a5546cfb417dab3dcaa799b1a1f5b8"}
Jul 28 19:38:41.845935 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:41.845911    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"
Jul 28 19:38:41.873470 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:41.873417    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz" podStartSLOduration=1.719086946 podStartE2EDuration="4.873398554s" totalImagesPullingTime="3.154311608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:38:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:38:41.872676261 +0000 UTC m=+547.460339210" watchObservedRunningTime="2026-07-28 19:38:41.873398554 +0000 UTC m=+547.461061499"
Jul 28 19:38:41.900729 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:41.900692    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg" podStartSLOduration=1.088171349 podStartE2EDuration="3.900679969s" totalImagesPullingTime="2.81250862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:38:41.900124549 +0000 UTC m=+547.487787495" watchObservedRunningTime="2026-07-28 19:38:41.900679969 +0000 UTC m=+547.488342913"
Jul 28 19:38:42.047211 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:42.047182    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:42.051513 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:42.051491    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:42.300245 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:42.300164    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:42.406575 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:42.406543    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:42.549766 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:42.549737    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:42.680886 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:42.680860    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:42.777447 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:42.777424    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:42.851131 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:42.851103    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:42.851809 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:42.851791    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz"
Jul 28 19:38:42.940370 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:42.940296    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:46.815833 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:46.815806    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-zxvjp"
Jul 28 19:38:47.823974 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:47.823950    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-fnmjf"
Jul 28 19:38:53.858232 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:53.858192    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6c85c6bd85-lqtpg"
Jul 28 19:38:57.593977 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:38:57.593934    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:06.156521 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:06.156484    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:32.421972 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:32.421929    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:32.565486 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:32.565456    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:32.623580 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:32.623554    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:34.742221 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:34.742193    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 19:39:34.743033 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:34.743006    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 19:39:47.951394 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:47.951360    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86"]
Jul 28 19:39:47.954682 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:47.954667    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86"
Jul 28 19:39:47.957193 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:47.957171    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 19:39:47.957608 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:47.957588    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-thhmk\""
Jul 28 19:39:47.957688 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:47.957591    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 19:39:47.963264 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:47.963239    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86"]
Jul 28 19:39:48.066794 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.066768    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-857hf\" (UniqueName: \"kubernetes.io/projected/cf19ca5b-3313-459c-9dba-08a23169d6f9-kube-api-access-857hf\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86\" (UID: \"cf19ca5b-3313-459c-9dba-08a23169d6f9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86"
Jul 28 19:39:48.066893 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.066812    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf19ca5b-3313-459c-9dba-08a23169d6f9-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86\" (UID: \"cf19ca5b-3313-459c-9dba-08a23169d6f9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86"
Jul 28 19:39:48.066934 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.066895    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf19ca5b-3313-459c-9dba-08a23169d6f9-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86\" (UID: \"cf19ca5b-3313-459c-9dba-08a23169d6f9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86"
Jul 28 19:39:48.167459 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.167432    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf19ca5b-3313-459c-9dba-08a23169d6f9-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86\" (UID: \"cf19ca5b-3313-459c-9dba-08a23169d6f9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86"
Jul 28 19:39:48.167633 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.167478    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-857hf\" (UniqueName: \"kubernetes.io/projected/cf19ca5b-3313-459c-9dba-08a23169d6f9-kube-api-access-857hf\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86\" (UID: \"cf19ca5b-3313-459c-9dba-08a23169d6f9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86"
Jul 28 19:39:48.167633 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.167538    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf19ca5b-3313-459c-9dba-08a23169d6f9-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86\" (UID: \"cf19ca5b-3313-459c-9dba-08a23169d6f9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86"
Jul 28 19:39:48.167791 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.167772    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf19ca5b-3313-459c-9dba-08a23169d6f9-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86\" (UID: \"cf19ca5b-3313-459c-9dba-08a23169d6f9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86"
Jul 28 19:39:48.167972 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.167954    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf19ca5b-3313-459c-9dba-08a23169d6f9-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86\" (UID: \"cf19ca5b-3313-459c-9dba-08a23169d6f9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86"
Jul 28 19:39:48.175814 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.175785    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-857hf\" (UniqueName: \"kubernetes.io/projected/cf19ca5b-3313-459c-9dba-08a23169d6f9-kube-api-access-857hf\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86\" (UID: \"cf19ca5b-3313-459c-9dba-08a23169d6f9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86"
Jul 28 19:39:48.264583 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.264532    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86"
Jul 28 19:39:48.389644 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.389621    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86"]
Jul 28 19:39:48.391555 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:39:48.391532    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf19ca5b_3313_459c_9dba_08a23169d6f9.slice/crio-e656ca3b5cca61b3919efb348ebf58ccd8dcd25bdc2315c67af4f080ca1c0963 WatchSource:0}: Error finding container e656ca3b5cca61b3919efb348ebf58ccd8dcd25bdc2315c67af4f080ca1c0963: Status 404 returned error can't find the container with id e656ca3b5cca61b3919efb348ebf58ccd8dcd25bdc2315c67af4f080ca1c0963
Jul 28 19:39:48.751467 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.751433    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4"]
Jul 28 19:39:48.754985 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.754966    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4"
Jul 28 19:39:48.761667 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.761639    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4"]
Jul 28 19:39:48.872760 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.872727    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4376f19b-d5bb-44f5-8cdc-eccea2adda49-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4\" (UID: \"4376f19b-d5bb-44f5-8cdc-eccea2adda49\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4"
Jul 28 19:39:48.872878 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.872768    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4376f19b-d5bb-44f5-8cdc-eccea2adda49-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4\" (UID: \"4376f19b-d5bb-44f5-8cdc-eccea2adda49\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4"
Jul 28 19:39:48.872878 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.872810    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpbhg\" (UniqueName: \"kubernetes.io/projected/4376f19b-d5bb-44f5-8cdc-eccea2adda49-kube-api-access-tpbhg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4\" (UID: \"4376f19b-d5bb-44f5-8cdc-eccea2adda49\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4"
Jul 28 19:39:48.973828 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.973801    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4376f19b-d5bb-44f5-8cdc-eccea2adda49-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4\" (UID: \"4376f19b-d5bb-44f5-8cdc-eccea2adda49\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4"
Jul 28 19:39:48.974166 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.973842    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4376f19b-d5bb-44f5-8cdc-eccea2adda49-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4\" (UID: \"4376f19b-d5bb-44f5-8cdc-eccea2adda49\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4"
Jul 28 19:39:48.974166 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.973877    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpbhg\" (UniqueName: \"kubernetes.io/projected/4376f19b-d5bb-44f5-8cdc-eccea2adda49-kube-api-access-tpbhg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4\" (UID: \"4376f19b-d5bb-44f5-8cdc-eccea2adda49\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4"
Jul 28 19:39:48.974240 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.974204    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4376f19b-d5bb-44f5-8cdc-eccea2adda49-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4\" (UID: \"4376f19b-d5bb-44f5-8cdc-eccea2adda49\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4"
Jul 28 19:39:48.974279 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.974249    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4376f19b-d5bb-44f5-8cdc-eccea2adda49-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4\" (UID: \"4376f19b-d5bb-44f5-8cdc-eccea2adda49\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4"
Jul 28 19:39:48.981809 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:48.981782    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpbhg\" (UniqueName: \"kubernetes.io/projected/4376f19b-d5bb-44f5-8cdc-eccea2adda49-kube-api-access-tpbhg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4\" (UID: \"4376f19b-d5bb-44f5-8cdc-eccea2adda49\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4"
Jul 28 19:39:49.065600 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.065575    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4"
Jul 28 19:39:49.155337 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.155304    2572 generic.go:356] "Generic (PLEG): container finished" podID="cf19ca5b-3313-459c-9dba-08a23169d6f9" containerID="0424c6ecf99642ea0fde87d66cdd7d0f6608924eda6eb9cc5c6d751f17b7d1dd" exitCode=0
Jul 28 19:39:49.155520 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.155344    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86" event={"ID":"cf19ca5b-3313-459c-9dba-08a23169d6f9","Type":"ContainerDied","Data":"0424c6ecf99642ea0fde87d66cdd7d0f6608924eda6eb9cc5c6d751f17b7d1dd"}
Jul 28 19:39:49.155632 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.155558    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86" event={"ID":"cf19ca5b-3313-459c-9dba-08a23169d6f9","Type":"ContainerStarted","Data":"e656ca3b5cca61b3919efb348ebf58ccd8dcd25bdc2315c67af4f080ca1c0963"}
Jul 28 19:39:49.155904 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.155890    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:39:49.191310 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.191285    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4"]
Jul 28 19:39:49.193545 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:39:49.193523    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4376f19b_d5bb_44f5_8cdc_eccea2adda49.slice/crio-f4148bbb7ced77f2abf57600d191abccbe2cc592f69d2a47206b5637309eeb63 WatchSource:0}: Error finding container f4148bbb7ced77f2abf57600d191abccbe2cc592f69d2a47206b5637309eeb63: Status 404 returned error can't find the container with id f4148bbb7ced77f2abf57600d191abccbe2cc592f69d2a47206b5637309eeb63
Jul 28 19:39:49.549992 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.549924    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9"]
Jul 28 19:39:49.553382 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.553365    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9"
Jul 28 19:39:49.560864 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.560843    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9"]
Jul 28 19:39:49.680588 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.680553    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl4xs\" (UniqueName: \"kubernetes.io/projected/6e3f64b5-e230-43af-946e-bec7e18d05e7-kube-api-access-cl4xs\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9\" (UID: \"6e3f64b5-e230-43af-946e-bec7e18d05e7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9"
Jul 28 19:39:49.680712 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.680599    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6e3f64b5-e230-43af-946e-bec7e18d05e7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9\" (UID: \"6e3f64b5-e230-43af-946e-bec7e18d05e7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9"
Jul 28 19:39:49.680712 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.680634    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6e3f64b5-e230-43af-946e-bec7e18d05e7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9\" (UID: \"6e3f64b5-e230-43af-946e-bec7e18d05e7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9"
Jul 28 19:39:49.781616 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.781590    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl4xs\" (UniqueName: \"kubernetes.io/projected/6e3f64b5-e230-43af-946e-bec7e18d05e7-kube-api-access-cl4xs\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9\" (UID: \"6e3f64b5-e230-43af-946e-bec7e18d05e7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9"
Jul 28 19:39:49.781731 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.781631    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6e3f64b5-e230-43af-946e-bec7e18d05e7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9\" (UID: \"6e3f64b5-e230-43af-946e-bec7e18d05e7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9"
Jul 28 19:39:49.781731 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.781685    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6e3f64b5-e230-43af-946e-bec7e18d05e7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9\" (UID: \"6e3f64b5-e230-43af-946e-bec7e18d05e7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9"
Jul 28 19:39:49.782028 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.782008    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6e3f64b5-e230-43af-946e-bec7e18d05e7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9\" (UID: \"6e3f64b5-e230-43af-946e-bec7e18d05e7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9"
Jul 28 19:39:49.782113 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.782063    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6e3f64b5-e230-43af-946e-bec7e18d05e7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9\" (UID: \"6e3f64b5-e230-43af-946e-bec7e18d05e7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9"
Jul 28 19:39:49.789652 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.789620    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl4xs\" (UniqueName: \"kubernetes.io/projected/6e3f64b5-e230-43af-946e-bec7e18d05e7-kube-api-access-cl4xs\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9\" (UID: \"6e3f64b5-e230-43af-946e-bec7e18d05e7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9"
Jul 28 19:39:49.863232 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.863205    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9"
Jul 28 19:39:49.956700 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.955972    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r"]
Jul 28 19:39:49.960344 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.960303    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r"
Jul 28 19:39:49.964907 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:49.964883    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r"]
Jul 28 19:39:50.003674 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.003654    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9"]
Jul 28 19:39:50.024441 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:39:50.024418    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e3f64b5_e230_43af_946e_bec7e18d05e7.slice/crio-64d463684899a891cfe62e771014d27ef5545c57cd00ed3b9010e77e48469900 WatchSource:0}: Error finding container 64d463684899a891cfe62e771014d27ef5545c57cd00ed3b9010e77e48469900: Status 404 returned error can't find the container with id 64d463684899a891cfe62e771014d27ef5545c57cd00ed3b9010e77e48469900
Jul 28 19:39:50.084278 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.084251    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77a8578c-ce3c-4fc7-9a4b-39382ad06958-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r\" (UID: \"77a8578c-ce3c-4fc7-9a4b-39382ad06958\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r"
Jul 28 19:39:50.084442 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.084293    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77a8578c-ce3c-4fc7-9a4b-39382ad06958-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r\" (UID: \"77a8578c-ce3c-4fc7-9a4b-39382ad06958\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r"
Jul 28 19:39:50.084442 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.084321    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hvfq\" (UniqueName: \"kubernetes.io/projected/77a8578c-ce3c-4fc7-9a4b-39382ad06958-kube-api-access-8hvfq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r\" (UID: \"77a8578c-ce3c-4fc7-9a4b-39382ad06958\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r"
Jul 28 19:39:50.162295 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.162258    2572 generic.go:356] "Generic (PLEG): container finished" podID="4376f19b-d5bb-44f5-8cdc-eccea2adda49" containerID="4a3e0a58fc8de30540a02130dbd4e66e34a7912109e5736593f1cdf5fbe11abf" exitCode=0
Jul 28 19:39:50.162418 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.162345    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4" event={"ID":"4376f19b-d5bb-44f5-8cdc-eccea2adda49","Type":"ContainerDied","Data":"4a3e0a58fc8de30540a02130dbd4e66e34a7912109e5736593f1cdf5fbe11abf"}
Jul 28 19:39:50.162418 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.162395    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4" event={"ID":"4376f19b-d5bb-44f5-8cdc-eccea2adda49","Type":"ContainerStarted","Data":"f4148bbb7ced77f2abf57600d191abccbe2cc592f69d2a47206b5637309eeb63"}
Jul 28 19:39:50.164212 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.164191    2572 generic.go:356] "Generic (PLEG): container finished" podID="cf19ca5b-3313-459c-9dba-08a23169d6f9" containerID="14671c3344b2ac39ecbb59b8c1781faa3f0b02d1739550b48bfbdeb818ffba16" exitCode=0
Jul 28 19:39:50.164331 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.164281    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86" event={"ID":"cf19ca5b-3313-459c-9dba-08a23169d6f9","Type":"ContainerDied","Data":"14671c3344b2ac39ecbb59b8c1781faa3f0b02d1739550b48bfbdeb818ffba16"}
Jul 28 19:39:50.165844 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.165818    2572 generic.go:356] "Generic (PLEG): container finished" podID="6e3f64b5-e230-43af-946e-bec7e18d05e7" containerID="4765928c5bece0244b51eb5eb4344a3336f255282d5e117ae579abc33e37426d" exitCode=0
Jul 28 19:39:50.165925 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.165871    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9" event={"ID":"6e3f64b5-e230-43af-946e-bec7e18d05e7","Type":"ContainerDied","Data":"4765928c5bece0244b51eb5eb4344a3336f255282d5e117ae579abc33e37426d"}
Jul 28 19:39:50.165925 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.165911    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9" event={"ID":"6e3f64b5-e230-43af-946e-bec7e18d05e7","Type":"ContainerStarted","Data":"64d463684899a891cfe62e771014d27ef5545c57cd00ed3b9010e77e48469900"}
Jul 28 19:39:50.185237 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.185205    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77a8578c-ce3c-4fc7-9a4b-39382ad06958-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r\" (UID: \"77a8578c-ce3c-4fc7-9a4b-39382ad06958\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r"
Jul 28 19:39:50.185377 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.185259    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77a8578c-ce3c-4fc7-9a4b-39382ad06958-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r\" (UID: \"77a8578c-ce3c-4fc7-9a4b-39382ad06958\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r"
Jul 28 19:39:50.185377 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.185292    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hvfq\" (UniqueName: \"kubernetes.io/projected/77a8578c-ce3c-4fc7-9a4b-39382ad06958-kube-api-access-8hvfq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r\" (UID: \"77a8578c-ce3c-4fc7-9a4b-39382ad06958\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r"
Jul 28 19:39:50.185798 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.185776    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77a8578c-ce3c-4fc7-9a4b-39382ad06958-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r\" (UID: \"77a8578c-ce3c-4fc7-9a4b-39382ad06958\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r"
Jul 28 19:39:50.185889 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.185827    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77a8578c-ce3c-4fc7-9a4b-39382ad06958-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r\" (UID: \"77a8578c-ce3c-4fc7-9a4b-39382ad06958\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r"
Jul 28 19:39:50.192362 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.192344    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hvfq\" (UniqueName: \"kubernetes.io/projected/77a8578c-ce3c-4fc7-9a4b-39382ad06958-kube-api-access-8hvfq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r\" (UID: \"77a8578c-ce3c-4fc7-9a4b-39382ad06958\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r"
Jul 28 19:39:50.274686 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.274663    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r"
Jul 28 19:39:50.403650 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.403625    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r"]
Jul 28 19:39:50.405249 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:39:50.405223    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77a8578c_ce3c_4fc7_9a4b_39382ad06958.slice/crio-80ab37b80fcc91da02bbe8f2897964161c51ba0cb128e4c7427f62b24d3ec6d6 WatchSource:0}: Error finding container 80ab37b80fcc91da02bbe8f2897964161c51ba0cb128e4c7427f62b24d3ec6d6: Status 404 returned error can't find the container with id 80ab37b80fcc91da02bbe8f2897964161c51ba0cb128e4c7427f62b24d3ec6d6
Jul 28 19:39:50.778962 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.778897    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f8bbf86d5-clnxs"]
Jul 28 19:39:50.784723 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.784703    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.792819 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.792777    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f8bbf86d5-clnxs"]
Jul 28 19:39:50.843791 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.843762    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:50.891147 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.891112    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/641696ee-422f-4d04-9ffd-d05f196614a4-console-serving-cert\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.891327 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.891163    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/641696ee-422f-4d04-9ffd-d05f196614a4-console-oauth-config\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.891327 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.891179    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/641696ee-422f-4d04-9ffd-d05f196614a4-service-ca\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.891327 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.891222    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-259t8\" (UniqueName: \"kubernetes.io/projected/641696ee-422f-4d04-9ffd-d05f196614a4-kube-api-access-259t8\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.891327 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.891247    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/641696ee-422f-4d04-9ffd-d05f196614a4-oauth-serving-cert\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.891327 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.891296    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/641696ee-422f-4d04-9ffd-d05f196614a4-trusted-ca-bundle\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.891327 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.891329    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/641696ee-422f-4d04-9ffd-d05f196614a4-console-config\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.992447 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.992414    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/641696ee-422f-4d04-9ffd-d05f196614a4-oauth-serving-cert\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.992592 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.992461    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/641696ee-422f-4d04-9ffd-d05f196614a4-trusted-ca-bundle\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.992592 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.992491    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/641696ee-422f-4d04-9ffd-d05f196614a4-console-config\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.992592 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.992534    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/641696ee-422f-4d04-9ffd-d05f196614a4-console-serving-cert\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.992592 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.992572    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/641696ee-422f-4d04-9ffd-d05f196614a4-console-oauth-config\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.992785 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.992593    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/641696ee-422f-4d04-9ffd-d05f196614a4-service-ca\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.992785 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.992634    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-259t8\" (UniqueName: \"kubernetes.io/projected/641696ee-422f-4d04-9ffd-d05f196614a4-kube-api-access-259t8\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.993187 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.993162    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/641696ee-422f-4d04-9ffd-d05f196614a4-oauth-serving-cert\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.993320 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.993268    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/641696ee-422f-4d04-9ffd-d05f196614a4-trusted-ca-bundle\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.993320 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.993292    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/641696ee-422f-4d04-9ffd-d05f196614a4-console-config\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.993320 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.993304    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/641696ee-422f-4d04-9ffd-d05f196614a4-service-ca\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.994857 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.994838    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/641696ee-422f-4d04-9ffd-d05f196614a4-console-serving-cert\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:50.994952 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:50.994857    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/641696ee-422f-4d04-9ffd-d05f196614a4-console-oauth-config\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:51.000814 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:51.000797    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-259t8\" (UniqueName: \"kubernetes.io/projected/641696ee-422f-4d04-9ffd-d05f196614a4-kube-api-access-259t8\") pod \"console-6f8bbf86d5-clnxs\" (UID: \"641696ee-422f-4d04-9ffd-d05f196614a4\") " pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:51.096227 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:51.096196    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:39:51.172756 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:51.172726    2572 generic.go:356] "Generic (PLEG): container finished" podID="4376f19b-d5bb-44f5-8cdc-eccea2adda49" containerID="733bd8865c1b3fc232cd261452f2944a90d2b0b8ecbe3ab0ec60fee2b081614e" exitCode=0
Jul 28 19:39:51.172896 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:51.172802    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4" event={"ID":"4376f19b-d5bb-44f5-8cdc-eccea2adda49","Type":"ContainerDied","Data":"733bd8865c1b3fc232cd261452f2944a90d2b0b8ecbe3ab0ec60fee2b081614e"}
Jul 28 19:39:51.175269 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:51.175248    2572 generic.go:356] "Generic (PLEG): container finished" podID="cf19ca5b-3313-459c-9dba-08a23169d6f9" containerID="199d73e3e5233e52e1194643584a14c3b2394e90238779fd5656433e00824412" exitCode=0
Jul 28 19:39:51.175361 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:51.175277    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86" event={"ID":"cf19ca5b-3313-459c-9dba-08a23169d6f9","Type":"ContainerDied","Data":"199d73e3e5233e52e1194643584a14c3b2394e90238779fd5656433e00824412"}
Jul 28 19:39:51.176887 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:51.176864    2572 generic.go:356] "Generic (PLEG): container finished" podID="77a8578c-ce3c-4fc7-9a4b-39382ad06958" containerID="635f82a5bdd6f695e5532391b869b815ad8ede729155ba45cf3cea3c10ec2869" exitCode=0
Jul 28 19:39:51.176969 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:51.176948    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r" event={"ID":"77a8578c-ce3c-4fc7-9a4b-39382ad06958","Type":"ContainerDied","Data":"635f82a5bdd6f695e5532391b869b815ad8ede729155ba45cf3cea3c10ec2869"}
Jul 28 19:39:51.177020 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:51.176987    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r" event={"ID":"77a8578c-ce3c-4fc7-9a4b-39382ad06958","Type":"ContainerStarted","Data":"80ab37b80fcc91da02bbe8f2897964161c51ba0cb128e4c7427f62b24d3ec6d6"}
Jul 28 19:39:51.179665 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:51.179301    2572 generic.go:356] "Generic (PLEG): container finished" podID="6e3f64b5-e230-43af-946e-bec7e18d05e7" containerID="749cb55a507cafc9994155d8f8c285730a21459b91cc8913c8240d8b47626153" exitCode=0
Jul 28 19:39:51.179665 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:51.179351    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9" event={"ID":"6e3f64b5-e230-43af-946e-bec7e18d05e7","Type":"ContainerDied","Data":"749cb55a507cafc9994155d8f8c285730a21459b91cc8913c8240d8b47626153"}
Jul 28 19:39:51.241681 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:51.237346    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f8bbf86d5-clnxs"]
Jul 28 19:39:51.249476 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:39:51.249446    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod641696ee_422f_4d04_9ffd_d05f196614a4.slice/crio-cfbbd7513bf64450e4d5e8632e70332af2e6adcfb2d3557624b280d596b0add9 WatchSource:0}: Error finding container cfbbd7513bf64450e4d5e8632e70332af2e6adcfb2d3557624b280d596b0add9: Status 404 returned error can't find the container with id cfbbd7513bf64450e4d5e8632e70332af2e6adcfb2d3557624b280d596b0add9
Jul 28 19:39:52.186459 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:52.186432    2572 generic.go:356] "Generic (PLEG): container finished" podID="6e3f64b5-e230-43af-946e-bec7e18d05e7" containerID="40413d24cc8d8faf38128ad748b9cf0527729ece97fbdd11d286e6cd68a958e4" exitCode=0
Jul 28 19:39:52.186807 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:52.186507    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9" event={"ID":"6e3f64b5-e230-43af-946e-bec7e18d05e7","Type":"ContainerDied","Data":"40413d24cc8d8faf38128ad748b9cf0527729ece97fbdd11d286e6cd68a958e4"}
Jul 28 19:39:52.188599 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:52.188572    2572 generic.go:356] "Generic (PLEG): container finished" podID="4376f19b-d5bb-44f5-8cdc-eccea2adda49" containerID="c63f329211919d7de23544d32a28e59135b66d1b51db16686fb198a0c8d8c42e" exitCode=0
Jul 28 19:39:52.188599 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:52.188597    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4" event={"ID":"4376f19b-d5bb-44f5-8cdc-eccea2adda49","Type":"ContainerDied","Data":"c63f329211919d7de23544d32a28e59135b66d1b51db16686fb198a0c8d8c42e"}
Jul 28 19:39:52.190354 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:52.190329    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f8bbf86d5-clnxs" event={"ID":"641696ee-422f-4d04-9ffd-d05f196614a4","Type":"ContainerStarted","Data":"c3d1ccfd4f8c3d7ae3df9c43752f4ccf89680f2d5a7a238e36d3f58d02c848be"}
Jul 28 19:39:52.190446 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:52.190373    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f8bbf86d5-clnxs" event={"ID":"641696ee-422f-4d04-9ffd-d05f196614a4","Type":"ContainerStarted","Data":"cfbbd7513bf64450e4d5e8632e70332af2e6adcfb2d3557624b280d596b0add9"}
Jul 28 19:39:52.218466 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:52.218424    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f8bbf86d5-clnxs" podStartSLOduration=2.218409324 podStartE2EDuration="2.218409324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:39:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:39:52.217569767 +0000 UTC m=+617.805232711" watchObservedRunningTime="2026-07-28 19:39:52.218409324 +0000 UTC m=+617.806072268"
Jul 28 19:39:52.360929 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:52.360908    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86"
Jul 28 19:39:52.399227 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:52.399196    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:52.504643 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:52.504588    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf19ca5b-3313-459c-9dba-08a23169d6f9-util\") pod \"cf19ca5b-3313-459c-9dba-08a23169d6f9\" (UID: \"cf19ca5b-3313-459c-9dba-08a23169d6f9\") "
Jul 28 19:39:52.504643 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:52.504616    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf19ca5b-3313-459c-9dba-08a23169d6f9-bundle\") pod \"cf19ca5b-3313-459c-9dba-08a23169d6f9\" (UID: \"cf19ca5b-3313-459c-9dba-08a23169d6f9\") "
Jul 28 19:39:52.504773 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:52.504656    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-857hf\" (UniqueName: \"kubernetes.io/projected/cf19ca5b-3313-459c-9dba-08a23169d6f9-kube-api-access-857hf\") pod \"cf19ca5b-3313-459c-9dba-08a23169d6f9\" (UID: \"cf19ca5b-3313-459c-9dba-08a23169d6f9\") "
Jul 28 19:39:52.505047 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:52.505022    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf19ca5b-3313-459c-9dba-08a23169d6f9-bundle" (OuterVolumeSpecName: "bundle") pod "cf19ca5b-3313-459c-9dba-08a23169d6f9" (UID: "cf19ca5b-3313-459c-9dba-08a23169d6f9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:39:52.506789 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:52.506764    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf19ca5b-3313-459c-9dba-08a23169d6f9-kube-api-access-857hf" (OuterVolumeSpecName: "kube-api-access-857hf") pod "cf19ca5b-3313-459c-9dba-08a23169d6f9" (UID: "cf19ca5b-3313-459c-9dba-08a23169d6f9"). InnerVolumeSpecName "kube-api-access-857hf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:39:52.510707 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:52.510670    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf19ca5b-3313-459c-9dba-08a23169d6f9-util" (OuterVolumeSpecName: "util") pod "cf19ca5b-3313-459c-9dba-08a23169d6f9" (UID: "cf19ca5b-3313-459c-9dba-08a23169d6f9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:39:52.605723 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:52.605699    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf19ca5b-3313-459c-9dba-08a23169d6f9-util\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:39:52.605723 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:52.605720    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf19ca5b-3313-459c-9dba-08a23169d6f9-bundle\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:39:52.605838 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:52.605729    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-857hf\" (UniqueName: \"kubernetes.io/projected/cf19ca5b-3313-459c-9dba-08a23169d6f9-kube-api-access-857hf\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:39:53.169269 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.169192    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:53.196909 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.196890    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86"
Jul 28 19:39:53.196909 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.196898    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86" event={"ID":"cf19ca5b-3313-459c-9dba-08a23169d6f9","Type":"ContainerDied","Data":"e656ca3b5cca61b3919efb348ebf58ccd8dcd25bdc2315c67af4f080ca1c0963"}
Jul 28 19:39:53.197356 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.196936    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e656ca3b5cca61b3919efb348ebf58ccd8dcd25bdc2315c67af4f080ca1c0963"
Jul 28 19:39:53.199181 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.199147    2572 generic.go:356] "Generic (PLEG): container finished" podID="77a8578c-ce3c-4fc7-9a4b-39382ad06958" containerID="d409e0f8db5886032288004923d28a6937de06bfc31dbb99a94b54966936effb" exitCode=0
Jul 28 19:39:53.199312 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.199248    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r" event={"ID":"77a8578c-ce3c-4fc7-9a4b-39382ad06958","Type":"ContainerDied","Data":"d409e0f8db5886032288004923d28a6937de06bfc31dbb99a94b54966936effb"}
Jul 28 19:39:53.219702 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.219676    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:53.334812 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.334791    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9"
Jul 28 19:39:53.415362 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.415341    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4"
Jul 28 19:39:53.513293 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.513244    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4376f19b-d5bb-44f5-8cdc-eccea2adda49-util\") pod \"4376f19b-d5bb-44f5-8cdc-eccea2adda49\" (UID: \"4376f19b-d5bb-44f5-8cdc-eccea2adda49\") "
Jul 28 19:39:53.513293 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.513277    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cl4xs\" (UniqueName: \"kubernetes.io/projected/6e3f64b5-e230-43af-946e-bec7e18d05e7-kube-api-access-cl4xs\") pod \"6e3f64b5-e230-43af-946e-bec7e18d05e7\" (UID: \"6e3f64b5-e230-43af-946e-bec7e18d05e7\") "
Jul 28 19:39:53.513293 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.513294    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6e3f64b5-e230-43af-946e-bec7e18d05e7-util\") pod \"6e3f64b5-e230-43af-946e-bec7e18d05e7\" (UID: \"6e3f64b5-e230-43af-946e-bec7e18d05e7\") "
Jul 28 19:39:53.513505 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.513321    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpbhg\" (UniqueName: \"kubernetes.io/projected/4376f19b-d5bb-44f5-8cdc-eccea2adda49-kube-api-access-tpbhg\") pod \"4376f19b-d5bb-44f5-8cdc-eccea2adda49\" (UID: \"4376f19b-d5bb-44f5-8cdc-eccea2adda49\") "
Jul 28 19:39:53.513505 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.513373    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6e3f64b5-e230-43af-946e-bec7e18d05e7-bundle\") pod \"6e3f64b5-e230-43af-946e-bec7e18d05e7\" (UID: \"6e3f64b5-e230-43af-946e-bec7e18d05e7\") "
Jul 28 19:39:53.513505 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.513423    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4376f19b-d5bb-44f5-8cdc-eccea2adda49-bundle\") pod \"4376f19b-d5bb-44f5-8cdc-eccea2adda49\" (UID: \"4376f19b-d5bb-44f5-8cdc-eccea2adda49\") "
Jul 28 19:39:53.513953 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.513873    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4376f19b-d5bb-44f5-8cdc-eccea2adda49-bundle" (OuterVolumeSpecName: "bundle") pod "4376f19b-d5bb-44f5-8cdc-eccea2adda49" (UID: "4376f19b-d5bb-44f5-8cdc-eccea2adda49"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:39:53.514140 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.514119    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e3f64b5-e230-43af-946e-bec7e18d05e7-bundle" (OuterVolumeSpecName: "bundle") pod "6e3f64b5-e230-43af-946e-bec7e18d05e7" (UID: "6e3f64b5-e230-43af-946e-bec7e18d05e7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:39:53.515339 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.515312    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4376f19b-d5bb-44f5-8cdc-eccea2adda49-kube-api-access-tpbhg" (OuterVolumeSpecName: "kube-api-access-tpbhg") pod "4376f19b-d5bb-44f5-8cdc-eccea2adda49" (UID: "4376f19b-d5bb-44f5-8cdc-eccea2adda49"). InnerVolumeSpecName "kube-api-access-tpbhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:39:53.515614 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.515594    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e3f64b5-e230-43af-946e-bec7e18d05e7-kube-api-access-cl4xs" (OuterVolumeSpecName: "kube-api-access-cl4xs") pod "6e3f64b5-e230-43af-946e-bec7e18d05e7" (UID: "6e3f64b5-e230-43af-946e-bec7e18d05e7"). InnerVolumeSpecName "kube-api-access-cl4xs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:39:53.518736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.518702    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4376f19b-d5bb-44f5-8cdc-eccea2adda49-util" (OuterVolumeSpecName: "util") pod "4376f19b-d5bb-44f5-8cdc-eccea2adda49" (UID: "4376f19b-d5bb-44f5-8cdc-eccea2adda49"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:39:53.518947 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.518926    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e3f64b5-e230-43af-946e-bec7e18d05e7-util" (OuterVolumeSpecName: "util") pod "6e3f64b5-e230-43af-946e-bec7e18d05e7" (UID: "6e3f64b5-e230-43af-946e-bec7e18d05e7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:39:53.614488 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.614457    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpbhg\" (UniqueName: \"kubernetes.io/projected/4376f19b-d5bb-44f5-8cdc-eccea2adda49-kube-api-access-tpbhg\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:39:53.614488 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.614487    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6e3f64b5-e230-43af-946e-bec7e18d05e7-bundle\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:39:53.614488 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.614496    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4376f19b-d5bb-44f5-8cdc-eccea2adda49-bundle\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:39:53.614641 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.614504    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4376f19b-d5bb-44f5-8cdc-eccea2adda49-util\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:39:53.614641 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.614511    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl4xs\" (UniqueName: \"kubernetes.io/projected/6e3f64b5-e230-43af-946e-bec7e18d05e7-kube-api-access-cl4xs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:39:53.614641 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:53.614520    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6e3f64b5-e230-43af-946e-bec7e18d05e7-util\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:39:54.207162 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:54.207138    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4"
Jul 28 19:39:54.207530 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:54.207135    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4" event={"ID":"4376f19b-d5bb-44f5-8cdc-eccea2adda49","Type":"ContainerDied","Data":"f4148bbb7ced77f2abf57600d191abccbe2cc592f69d2a47206b5637309eeb63"}
Jul 28 19:39:54.207530 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:54.207268    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4148bbb7ced77f2abf57600d191abccbe2cc592f69d2a47206b5637309eeb63"
Jul 28 19:39:54.209799 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:54.209769    2572 generic.go:356] "Generic (PLEG): container finished" podID="77a8578c-ce3c-4fc7-9a4b-39382ad06958" containerID="b42e31143c495d2ad2184bd0b6c81289b1ef337d7f8758a127a73f99fd24d2b0" exitCode=0
Jul 28 19:39:54.209969 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:54.209845    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r" event={"ID":"77a8578c-ce3c-4fc7-9a4b-39382ad06958","Type":"ContainerDied","Data":"b42e31143c495d2ad2184bd0b6c81289b1ef337d7f8758a127a73f99fd24d2b0"}
Jul 28 19:39:54.212093 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:54.212049    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9" event={"ID":"6e3f64b5-e230-43af-946e-bec7e18d05e7","Type":"ContainerDied","Data":"64d463684899a891cfe62e771014d27ef5545c57cd00ed3b9010e77e48469900"}
Jul 28 19:39:54.212194 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:54.212107    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64d463684899a891cfe62e771014d27ef5545c57cd00ed3b9010e77e48469900"
Jul 28 19:39:54.212194 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:54.212113    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9"
Jul 28 19:39:55.345906 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:55.345883    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r"
Jul 28 19:39:55.427596 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:55.427574    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77a8578c-ce3c-4fc7-9a4b-39382ad06958-bundle\") pod \"77a8578c-ce3c-4fc7-9a4b-39382ad06958\" (UID: \"77a8578c-ce3c-4fc7-9a4b-39382ad06958\") "
Jul 28 19:39:55.427748 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:55.427729    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hvfq\" (UniqueName: \"kubernetes.io/projected/77a8578c-ce3c-4fc7-9a4b-39382ad06958-kube-api-access-8hvfq\") pod \"77a8578c-ce3c-4fc7-9a4b-39382ad06958\" (UID: \"77a8578c-ce3c-4fc7-9a4b-39382ad06958\") "
Jul 28 19:39:55.427792 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:55.427780    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77a8578c-ce3c-4fc7-9a4b-39382ad06958-util\") pod \"77a8578c-ce3c-4fc7-9a4b-39382ad06958\" (UID: \"77a8578c-ce3c-4fc7-9a4b-39382ad06958\") "
Jul 28 19:39:55.428181 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:55.428152    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77a8578c-ce3c-4fc7-9a4b-39382ad06958-bundle" (OuterVolumeSpecName: "bundle") pod "77a8578c-ce3c-4fc7-9a4b-39382ad06958" (UID: "77a8578c-ce3c-4fc7-9a4b-39382ad06958"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:39:55.429709 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:55.429683    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77a8578c-ce3c-4fc7-9a4b-39382ad06958-kube-api-access-8hvfq" (OuterVolumeSpecName: "kube-api-access-8hvfq") pod "77a8578c-ce3c-4fc7-9a4b-39382ad06958" (UID: "77a8578c-ce3c-4fc7-9a4b-39382ad06958"). InnerVolumeSpecName "kube-api-access-8hvfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:39:55.433113 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:55.433089    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77a8578c-ce3c-4fc7-9a4b-39382ad06958-util" (OuterVolumeSpecName: "util") pod "77a8578c-ce3c-4fc7-9a4b-39382ad06958" (UID: "77a8578c-ce3c-4fc7-9a4b-39382ad06958"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:39:55.528573 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:55.528523    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hvfq\" (UniqueName: \"kubernetes.io/projected/77a8578c-ce3c-4fc7-9a4b-39382ad06958-kube-api-access-8hvfq\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:39:55.528573 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:55.528542    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77a8578c-ce3c-4fc7-9a4b-39382ad06958-util\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:39:55.528573 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:55.528552    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77a8578c-ce3c-4fc7-9a4b-39382ad06958-bundle\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:39:56.225271 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:56.225240    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r"
Jul 28 19:39:56.225435 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:56.225268    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r" event={"ID":"77a8578c-ce3c-4fc7-9a4b-39382ad06958","Type":"ContainerDied","Data":"80ab37b80fcc91da02bbe8f2897964161c51ba0cb128e4c7427f62b24d3ec6d6"}
Jul 28 19:39:56.225435 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:39:56.225320    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80ab37b80fcc91da02bbe8f2897964161c51ba0cb128e4c7427f62b24d3ec6d6"
Jul 28 19:40:01.097328 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:01.097237    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:40:01.097328 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:01.097310    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:40:01.101993 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:01.101966    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:40:01.254952 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:01.254922    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f8bbf86d5-clnxs"
Jul 28 19:40:01.304125 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:01.304101    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55b95f6b68-rs9zz"]
Jul 28 19:40:01.419828 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:01.419750    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:01.587674 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:01.587644    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:01.689439 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:01.689378    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:01.932500 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:01.932470    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:05.329637 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:05.329587    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:06.128048 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:06.128001    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:09.394943 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:09.394907    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:09.744682 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:09.744604    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:11.354626 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:11.354565    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:12.556375 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:12.556342    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:12.658913 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:12.658885    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:14.371035 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:14.371004    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:17.531346 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:17.531311    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:17.634825 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:17.634794    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:22.930887 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:22.930853    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:22.980817 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:22.980784    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:23.336058 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:23.336030    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:23.438863 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:23.438838    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:25.854857 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:25.854823    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:26.022726 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.022701    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:26.323611 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.323578    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-55b95f6b68-rs9zz" podUID="f916ea54-9231-4f6b-a810-a6a98a8fcaae" containerName="console" containerID="cri-o://1a3963f023827db201c5eb27325c45c0a3f5ce78ca3070f3d2abc7c25cbddbe1" gracePeriod=15
Jul 28 19:40:26.343126 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.343099    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:26.390241 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.390221    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:26.561645 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.561620    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:40:26.659565 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.659504    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-service-ca\") pod \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") "
Jul 28 19:40:26.659565 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.659550    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-console-config\") pod \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") "
Jul 28 19:40:26.659733 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.659586    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f916ea54-9231-4f6b-a810-a6a98a8fcaae-console-oauth-config\") pod \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") "
Jul 28 19:40:26.659733 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.659620    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f916ea54-9231-4f6b-a810-a6a98a8fcaae-console-serving-cert\") pod \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") "
Jul 28 19:40:26.659733 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.659658    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-oauth-serving-cert\") pod \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") "
Jul 28 19:40:26.659733 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.659702    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7bcv\" (UniqueName: \"kubernetes.io/projected/f916ea54-9231-4f6b-a810-a6a98a8fcaae-kube-api-access-h7bcv\") pod \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") "
Jul 28 19:40:26.659733 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.659728    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-trusted-ca-bundle\") pod \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\" (UID: \"f916ea54-9231-4f6b-a810-a6a98a8fcaae\") "
Jul 28 19:40:26.659979 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.659791    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-service-ca" (OuterVolumeSpecName: "service-ca") pod "f916ea54-9231-4f6b-a810-a6a98a8fcaae" (UID: "f916ea54-9231-4f6b-a810-a6a98a8fcaae"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:40:26.659979 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.659963    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-service-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:40:26.660122 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.660022    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-console-config" (OuterVolumeSpecName: "console-config") pod "f916ea54-9231-4f6b-a810-a6a98a8fcaae" (UID: "f916ea54-9231-4f6b-a810-a6a98a8fcaae"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:40:26.660122 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.660105    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f916ea54-9231-4f6b-a810-a6a98a8fcaae" (UID: "f916ea54-9231-4f6b-a810-a6a98a8fcaae"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:40:26.660223 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.660168    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f916ea54-9231-4f6b-a810-a6a98a8fcaae" (UID: "f916ea54-9231-4f6b-a810-a6a98a8fcaae"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:40:26.661758 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.661727    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f916ea54-9231-4f6b-a810-a6a98a8fcaae-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f916ea54-9231-4f6b-a810-a6a98a8fcaae" (UID: "f916ea54-9231-4f6b-a810-a6a98a8fcaae"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:40:26.661871 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.661847    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f916ea54-9231-4f6b-a810-a6a98a8fcaae-kube-api-access-h7bcv" (OuterVolumeSpecName: "kube-api-access-h7bcv") pod "f916ea54-9231-4f6b-a810-a6a98a8fcaae" (UID: "f916ea54-9231-4f6b-a810-a6a98a8fcaae"). InnerVolumeSpecName "kube-api-access-h7bcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:40:26.661954 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.661933    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f916ea54-9231-4f6b-a810-a6a98a8fcaae-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f916ea54-9231-4f6b-a810-a6a98a8fcaae" (UID: "f916ea54-9231-4f6b-a810-a6a98a8fcaae"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:40:26.761011 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.760987    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-oauth-serving-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:40:26.761011 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.761007    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7bcv\" (UniqueName: \"kubernetes.io/projected/f916ea54-9231-4f6b-a810-a6a98a8fcaae-kube-api-access-h7bcv\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:40:26.761166 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.761018    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-trusted-ca-bundle\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:40:26.761166 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.761028    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f916ea54-9231-4f6b-a810-a6a98a8fcaae-console-config\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:40:26.761166 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.761036    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f916ea54-9231-4f6b-a810-a6a98a8fcaae-console-oauth-config\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:40:26.761166 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:26.761044    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f916ea54-9231-4f6b-a810-a6a98a8fcaae-console-serving-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:40:27.383093 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:27.383033    2572 generic.go:356] "Generic (PLEG): container finished" podID="f916ea54-9231-4f6b-a810-a6a98a8fcaae" containerID="1a3963f023827db201c5eb27325c45c0a3f5ce78ca3070f3d2abc7c25cbddbe1" exitCode=0
Jul 28 19:40:27.383494 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:27.383122    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55b95f6b68-rs9zz"
Jul 28 19:40:27.383494 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:27.383137    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55b95f6b68-rs9zz" event={"ID":"f916ea54-9231-4f6b-a810-a6a98a8fcaae","Type":"ContainerDied","Data":"1a3963f023827db201c5eb27325c45c0a3f5ce78ca3070f3d2abc7c25cbddbe1"}
Jul 28 19:40:27.383494 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:27.383200    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55b95f6b68-rs9zz" event={"ID":"f916ea54-9231-4f6b-a810-a6a98a8fcaae","Type":"ContainerDied","Data":"6a3ad15dbb285c9fd3a2ce563f953acfb3ab567aad37b63d2d4f8125d7a50647"}
Jul 28 19:40:27.383494 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:27.383225    2572 scope.go:117] "RemoveContainer" containerID="1a3963f023827db201c5eb27325c45c0a3f5ce78ca3070f3d2abc7c25cbddbe1"
Jul 28 19:40:27.393010 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:27.392990    2572 scope.go:117] "RemoveContainer" containerID="1a3963f023827db201c5eb27325c45c0a3f5ce78ca3070f3d2abc7c25cbddbe1"
Jul 28 19:40:27.393340 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:40:27.393318    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a3963f023827db201c5eb27325c45c0a3f5ce78ca3070f3d2abc7c25cbddbe1\": container with ID starting with 1a3963f023827db201c5eb27325c45c0a3f5ce78ca3070f3d2abc7c25cbddbe1 not found: ID does not exist" containerID="1a3963f023827db201c5eb27325c45c0a3f5ce78ca3070f3d2abc7c25cbddbe1"
Jul 28 19:40:27.393392 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:27.393350    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a3963f023827db201c5eb27325c45c0a3f5ce78ca3070f3d2abc7c25cbddbe1"} err="failed to get container status \"1a3963f023827db201c5eb27325c45c0a3f5ce78ca3070f3d2abc7c25cbddbe1\": rpc error: code = NotFound desc = could not find container \"1a3963f023827db201c5eb27325c45c0a3f5ce78ca3070f3d2abc7c25cbddbe1\": container with ID starting with 1a3963f023827db201c5eb27325c45c0a3f5ce78ca3070f3d2abc7c25cbddbe1 not found: ID does not exist"
Jul 28 19:40:27.400192 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:27.400167    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55b95f6b68-rs9zz"]
Jul 28 19:40:27.403569 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:27.403543    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55b95f6b68-rs9zz"]
Jul 28 19:40:28.864855 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:28.864821    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f916ea54-9231-4f6b-a810-a6a98a8fcaae" path="/var/lib/kubelet/pods/f916ea54-9231-4f6b-a810-a6a98a8fcaae/volumes"
Jul 28 19:40:30.525361 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:30.525330    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:30.836493 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:30.836461    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:37.144240 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:37.144210    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:37.244517 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:37.244486    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:40.916322 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:40.916280    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:40.976636 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:40.976603    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:41.091897 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:41.091875    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:41.152762 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:41.152739    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:42.370259 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:42.370230    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:44.892474 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:44.892439    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:44.943619 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:44.943593    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:50.299212 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:50.299123    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:50.403491 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:40:50.403454    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:06.716345 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:06.716307    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:06.774085 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:06.774053    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:17.036840 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.036805    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:17.841047 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841014    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"]
Jul 28 19:41:17.841435 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841414    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e3f64b5-e230-43af-946e-bec7e18d05e7" containerName="pull"
Jul 28 19:41:17.841483 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841441    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e3f64b5-e230-43af-946e-bec7e18d05e7" containerName="pull"
Jul 28 19:41:17.841483 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841459    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e3f64b5-e230-43af-946e-bec7e18d05e7" containerName="util"
Jul 28 19:41:17.841483 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841468    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e3f64b5-e230-43af-946e-bec7e18d05e7" containerName="util"
Jul 28 19:41:17.841483 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841481    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf19ca5b-3313-459c-9dba-08a23169d6f9" containerName="util"
Jul 28 19:41:17.841610 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841489    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf19ca5b-3313-459c-9dba-08a23169d6f9" containerName="util"
Jul 28 19:41:17.841610 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841514    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e3f64b5-e230-43af-946e-bec7e18d05e7" containerName="extract"
Jul 28 19:41:17.841610 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841522    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e3f64b5-e230-43af-946e-bec7e18d05e7" containerName="extract"
Jul 28 19:41:17.841610 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841532    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77a8578c-ce3c-4fc7-9a4b-39382ad06958" containerName="pull"
Jul 28 19:41:17.841610 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841540    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="77a8578c-ce3c-4fc7-9a4b-39382ad06958" containerName="pull"
Jul 28 19:41:17.841610 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841553    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f916ea54-9231-4f6b-a810-a6a98a8fcaae" containerName="console"
Jul 28 19:41:17.841610 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841558    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f916ea54-9231-4f6b-a810-a6a98a8fcaae" containerName="console"
Jul 28 19:41:17.841610 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841565    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77a8578c-ce3c-4fc7-9a4b-39382ad06958" containerName="util"
Jul 28 19:41:17.841610 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841570    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="77a8578c-ce3c-4fc7-9a4b-39382ad06958" containerName="util"
Jul 28 19:41:17.841610 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841578    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf19ca5b-3313-459c-9dba-08a23169d6f9" containerName="extract"
Jul 28 19:41:17.841610 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841582    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf19ca5b-3313-459c-9dba-08a23169d6f9" containerName="extract"
Jul 28 19:41:17.841610 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841590    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4376f19b-d5bb-44f5-8cdc-eccea2adda49" containerName="util"
Jul 28 19:41:17.841610 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841595    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4376f19b-d5bb-44f5-8cdc-eccea2adda49" containerName="util"
Jul 28 19:41:17.841610 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841600    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4376f19b-d5bb-44f5-8cdc-eccea2adda49" containerName="pull"
Jul 28 19:41:17.841610 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841605    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4376f19b-d5bb-44f5-8cdc-eccea2adda49" containerName="pull"
Jul 28 19:41:17.841610 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841612    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf19ca5b-3313-459c-9dba-08a23169d6f9" containerName="pull"
Jul 28 19:41:17.841610 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841619    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf19ca5b-3313-459c-9dba-08a23169d6f9" containerName="pull"
Jul 28 19:41:17.842137 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841627    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4376f19b-d5bb-44f5-8cdc-eccea2adda49" containerName="extract"
Jul 28 19:41:17.842137 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841632    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4376f19b-d5bb-44f5-8cdc-eccea2adda49" containerName="extract"
Jul 28 19:41:17.842137 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841639    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77a8578c-ce3c-4fc7-9a4b-39382ad06958" containerName="extract"
Jul 28 19:41:17.842137 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841644    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="77a8578c-ce3c-4fc7-9a4b-39382ad06958" containerName="extract"
Jul 28 19:41:17.842137 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841706    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4376f19b-d5bb-44f5-8cdc-eccea2adda49" containerName="extract"
Jul 28 19:41:17.842137 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841715    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="77a8578c-ce3c-4fc7-9a4b-39382ad06958" containerName="extract"
Jul 28 19:41:17.842137 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841724    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f916ea54-9231-4f6b-a810-a6a98a8fcaae" containerName="console"
Jul 28 19:41:17.842137 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841735    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf19ca5b-3313-459c-9dba-08a23169d6f9" containerName="extract"
Jul 28 19:41:17.842137 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.841743    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e3f64b5-e230-43af-946e-bec7e18d05e7" containerName="extract"
Jul 28 19:41:17.844994 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.844974    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:17.847550 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.847526    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-bgg8d\""
Jul 28 19:41:17.855569 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.855541    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"]
Jul 28 19:41:17.871531 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.871504    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:17.931716 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.931693    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36b15eb7-edfa-4803-8270-332cafadae6c-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:17.931821 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.931721    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36b15eb7-edfa-4803-8270-332cafadae6c-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:17.931821 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.931750    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36b15eb7-edfa-4803-8270-332cafadae6c-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:17.931944 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.931827    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36b15eb7-edfa-4803-8270-332cafadae6c-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:17.931944 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.931874    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbfrs\" (UniqueName: \"kubernetes.io/projected/36b15eb7-edfa-4803-8270-332cafadae6c-kube-api-access-fbfrs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:17.931944 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.931923    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36b15eb7-edfa-4803-8270-332cafadae6c-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:17.932049 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.931950    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36b15eb7-edfa-4803-8270-332cafadae6c-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:17.932049 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.931997    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36b15eb7-edfa-4803-8270-332cafadae6c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:17.932049 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:17.932020    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36b15eb7-edfa-4803-8270-332cafadae6c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.033030 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.033005    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36b15eb7-edfa-4803-8270-332cafadae6c-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.033133 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.033037    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36b15eb7-edfa-4803-8270-332cafadae6c-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.033133 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.033063    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbfrs\" (UniqueName: \"kubernetes.io/projected/36b15eb7-edfa-4803-8270-332cafadae6c-kube-api-access-fbfrs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.033133 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.033114    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36b15eb7-edfa-4803-8270-332cafadae6c-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.033262 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.033141    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36b15eb7-edfa-4803-8270-332cafadae6c-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.033262 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.033171    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36b15eb7-edfa-4803-8270-332cafadae6c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.033262 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.033195    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36b15eb7-edfa-4803-8270-332cafadae6c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.033262 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.033232    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36b15eb7-edfa-4803-8270-332cafadae6c-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.033262 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.033255    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36b15eb7-edfa-4803-8270-332cafadae6c-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.033534 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.033389    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36b15eb7-edfa-4803-8270-332cafadae6c-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.033534 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.033435    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36b15eb7-edfa-4803-8270-332cafadae6c-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.034029 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.033967    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36b15eb7-edfa-4803-8270-332cafadae6c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.034160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.034131    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36b15eb7-edfa-4803-8270-332cafadae6c-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.034160 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.034138    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36b15eb7-edfa-4803-8270-332cafadae6c-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.035877 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.035849    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36b15eb7-edfa-4803-8270-332cafadae6c-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.035955 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.035882    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36b15eb7-edfa-4803-8270-332cafadae6c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.040264 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.040243    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36b15eb7-edfa-4803-8270-332cafadae6c-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.040609 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.040470    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbfrs\" (UniqueName: \"kubernetes.io/projected/36b15eb7-edfa-4803-8270-332cafadae6c-kube-api-access-fbfrs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c5xr9\" (UID: \"36b15eb7-edfa-4803-8270-332cafadae6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.090101 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.090055    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:18.157235 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.157183    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:18.290000 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.289970    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"]
Jul 28 19:41:18.291745 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:41:18.291721    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36b15eb7_edfa_4803_8270_332cafadae6c.slice/crio-6c5041cbab02592ce979f800fce461512afc927b44b74288954d28cf88775025 WatchSource:0}: Error finding container 6c5041cbab02592ce979f800fce461512afc927b44b74288954d28cf88775025: Status 404 returned error can't find the container with id 6c5041cbab02592ce979f800fce461512afc927b44b74288954d28cf88775025
Jul 28 19:41:18.301456 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.301423    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:41:18.301539 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.301487    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:41:18.301539 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.301516    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:41:18.637182 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.637143    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9" event={"ID":"36b15eb7-edfa-4803-8270-332cafadae6c","Type":"ContainerStarted","Data":"b0148a088fc5dad74d3fca7c59ed746b4ebb273bbadf7d5bccdd18eb9cad6742"}
Jul 28 19:41:18.637360 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.637243    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9" event={"ID":"36b15eb7-edfa-4803-8270-332cafadae6c","Type":"ContainerStarted","Data":"6c5041cbab02592ce979f800fce461512afc927b44b74288954d28cf88775025"}
Jul 28 19:41:18.657579 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:18.657531    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9" podStartSLOduration=1.65751688 podStartE2EDuration="1.65751688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:41:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:41:18.656614937 +0000 UTC m=+704.244277883" watchObservedRunningTime="2026-07-28 19:41:18.65751688 +0000 UTC m=+704.245179825"
Jul 28 19:41:19.157601 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:19.157571    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:19.161761 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:19.161739    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:19.642391 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:19.642352    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:19.643222 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:19.643197    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:41:33.472706 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:33.472620    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:33.579841 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:33.579815    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:33.683458 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:33.683410    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:34.281296 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.281250    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-2k9mm"]
Jul 28 19:41:34.284863 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.284834    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2k9mm"
Jul 28 19:41:34.287560 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.287537    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 19:41:34.288455 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.288422    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-494lb\""
Jul 28 19:41:34.288585 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.288527    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 19:41:34.291081 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.291003    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-2k9mm"]
Jul 28 19:41:34.310021 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.309997    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:34.360578 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.360556    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdthm\" (UniqueName: \"kubernetes.io/projected/c968f60c-bdbe-494f-a5c5-4ada24581c59-kube-api-access-rdthm\") pod \"authorino-58cd955675-2k9mm\" (UID: \"c968f60c-bdbe-494f-a5c5-4ada24581c59\") " pod="kuadrant-system/authorino-58cd955675-2k9mm"
Jul 28 19:41:34.366711 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.366693    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:34.461005 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.460982    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdthm\" (UniqueName: \"kubernetes.io/projected/c968f60c-bdbe-494f-a5c5-4ada24581c59-kube-api-access-rdthm\") pod \"authorino-58cd955675-2k9mm\" (UID: \"c968f60c-bdbe-494f-a5c5-4ada24581c59\") " pod="kuadrant-system/authorino-58cd955675-2k9mm"
Jul 28 19:41:34.468735 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.468711    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdthm\" (UniqueName: \"kubernetes.io/projected/c968f60c-bdbe-494f-a5c5-4ada24581c59-kube-api-access-rdthm\") pod \"authorino-58cd955675-2k9mm\" (UID: \"c968f60c-bdbe-494f-a5c5-4ada24581c59\") " pod="kuadrant-system/authorino-58cd955675-2k9mm"
Jul 28 19:41:34.514181 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.514160    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-frxsh"]
Jul 28 19:41:34.517368 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.517354    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-frxsh"
Jul 28 19:41:34.526266 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.526248    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-frxsh"]
Jul 28 19:41:34.562253 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.562235    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfb9g\" (UniqueName: \"kubernetes.io/projected/f63d08ab-939f-4062-a590-daf9bb061c06-kube-api-access-wfb9g\") pod \"authorino-75d556bd77-frxsh\" (UID: \"f63d08ab-939f-4062-a590-daf9bb061c06\") " pod="kuadrant-system/authorino-75d556bd77-frxsh"
Jul 28 19:41:34.597768 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.597747    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2k9mm"
Jul 28 19:41:34.662687 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.662660    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfb9g\" (UniqueName: \"kubernetes.io/projected/f63d08ab-939f-4062-a590-daf9bb061c06-kube-api-access-wfb9g\") pod \"authorino-75d556bd77-frxsh\" (UID: \"f63d08ab-939f-4062-a590-daf9bb061c06\") " pod="kuadrant-system/authorino-75d556bd77-frxsh"
Jul 28 19:41:34.670304 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.670280    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfb9g\" (UniqueName: \"kubernetes.io/projected/f63d08ab-939f-4062-a590-daf9bb061c06-kube-api-access-wfb9g\") pod \"authorino-75d556bd77-frxsh\" (UID: \"f63d08ab-939f-4062-a590-daf9bb061c06\") " pod="kuadrant-system/authorino-75d556bd77-frxsh"
Jul 28 19:41:34.725686 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.725659    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-2k9mm"]
Jul 28 19:41:34.726979 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:41:34.726956    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc968f60c_bdbe_494f_a5c5_4ada24581c59.slice/crio-0743eff00e5bc6b7103d18f080de530f3ad76254f5134e5300ee98c720682ec8 WatchSource:0}: Error finding container 0743eff00e5bc6b7103d18f080de530f3ad76254f5134e5300ee98c720682ec8: Status 404 returned error can't find the container with id 0743eff00e5bc6b7103d18f080de530f3ad76254f5134e5300ee98c720682ec8
Jul 28 19:41:34.826134 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.826088    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-frxsh"
Jul 28 19:41:34.955712 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:34.955690    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-frxsh"]
Jul 28 19:41:34.956976 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:41:34.956949    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf63d08ab_939f_4062_a590_daf9bb061c06.slice/crio-df5aef8211a417fe5abe06a763ea4f4dbf4a78dfce0c9a777894d0b11ae30380 WatchSource:0}: Error finding container df5aef8211a417fe5abe06a763ea4f4dbf4a78dfce0c9a777894d0b11ae30380: Status 404 returned error can't find the container with id df5aef8211a417fe5abe06a763ea4f4dbf4a78dfce0c9a777894d0b11ae30380
Jul 28 19:41:35.731431 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:35.731393    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-frxsh" event={"ID":"f63d08ab-939f-4062-a590-daf9bb061c06","Type":"ContainerStarted","Data":"df5aef8211a417fe5abe06a763ea4f4dbf4a78dfce0c9a777894d0b11ae30380"}
Jul 28 19:41:35.734666 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:35.734612    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2k9mm" event={"ID":"c968f60c-bdbe-494f-a5c5-4ada24581c59","Type":"ContainerStarted","Data":"0743eff00e5bc6b7103d18f080de530f3ad76254f5134e5300ee98c720682ec8"}
Jul 28 19:41:38.752637 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:38.752598    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-frxsh" event={"ID":"f63d08ab-939f-4062-a590-daf9bb061c06","Type":"ContainerStarted","Data":"c11790f82627f0957a176d4e65031f349b103fd3620f57f9854040ee255cdf72"}
Jul 28 19:41:38.754318 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:38.754288    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2k9mm" event={"ID":"c968f60c-bdbe-494f-a5c5-4ada24581c59","Type":"ContainerStarted","Data":"71a71fbd077ce90eb4407605b347850051482cfe3bb53f06ee864acfb6111fbe"}
Jul 28 19:41:38.766619 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:38.766563    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-frxsh" podStartSLOduration=1.541787087 podStartE2EDuration="4.766544906s" totalImagesPullingTime="3.224757819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:41:38.766451399 +0000 UTC m=+724.354114344" watchObservedRunningTime="2026-07-28 19:41:38.766544906 +0000 UTC m=+724.354207852"
Jul 28 19:41:38.780416 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:38.780369    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-2k9mm" podStartSLOduration=1.314228574 podStartE2EDuration="4.780354511s" totalImagesPullingTime="3.466125937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:41:38.778617369 +0000 UTC m=+724.366280314" watchObservedRunningTime="2026-07-28 19:41:38.780354511 +0000 UTC m=+724.368017455"
Jul 28 19:41:38.800354 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:38.800314    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-2k9mm"]
Jul 28 19:41:40.765124 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:40.765050    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-2k9mm" podUID="c968f60c-bdbe-494f-a5c5-4ada24581c59" containerName="authorino" containerID="cri-o://71a71fbd077ce90eb4407605b347850051482cfe3bb53f06ee864acfb6111fbe" gracePeriod=30
Jul 28 19:41:41.003165 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:41.003145    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2k9mm"
Jul 28 19:41:41.118471 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:41.118447    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdthm\" (UniqueName: \"kubernetes.io/projected/c968f60c-bdbe-494f-a5c5-4ada24581c59-kube-api-access-rdthm\") pod \"c968f60c-bdbe-494f-a5c5-4ada24581c59\" (UID: \"c968f60c-bdbe-494f-a5c5-4ada24581c59\") "
Jul 28 19:41:41.120348 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:41.120315    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c968f60c-bdbe-494f-a5c5-4ada24581c59-kube-api-access-rdthm" (OuterVolumeSpecName: "kube-api-access-rdthm") pod "c968f60c-bdbe-494f-a5c5-4ada24581c59" (UID: "c968f60c-bdbe-494f-a5c5-4ada24581c59"). InnerVolumeSpecName "kube-api-access-rdthm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:41:41.219415 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:41.219388    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdthm\" (UniqueName: \"kubernetes.io/projected/c968f60c-bdbe-494f-a5c5-4ada24581c59-kube-api-access-rdthm\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:41:41.771672 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:41.771639    2572 generic.go:356] "Generic (PLEG): container finished" podID="c968f60c-bdbe-494f-a5c5-4ada24581c59" containerID="71a71fbd077ce90eb4407605b347850051482cfe3bb53f06ee864acfb6111fbe" exitCode=0
Jul 28 19:41:41.772038 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:41.771687    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2k9mm"
Jul 28 19:41:41.772038 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:41.771718    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2k9mm" event={"ID":"c968f60c-bdbe-494f-a5c5-4ada24581c59","Type":"ContainerDied","Data":"71a71fbd077ce90eb4407605b347850051482cfe3bb53f06ee864acfb6111fbe"}
Jul 28 19:41:41.772038 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:41.771753    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2k9mm" event={"ID":"c968f60c-bdbe-494f-a5c5-4ada24581c59","Type":"ContainerDied","Data":"0743eff00e5bc6b7103d18f080de530f3ad76254f5134e5300ee98c720682ec8"}
Jul 28 19:41:41.772038 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:41.771770    2572 scope.go:117] "RemoveContainer" containerID="71a71fbd077ce90eb4407605b347850051482cfe3bb53f06ee864acfb6111fbe"
Jul 28 19:41:41.784422 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:41.784314    2572 scope.go:117] "RemoveContainer" containerID="71a71fbd077ce90eb4407605b347850051482cfe3bb53f06ee864acfb6111fbe"
Jul 28 19:41:41.784749 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:41:41.784687    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71a71fbd077ce90eb4407605b347850051482cfe3bb53f06ee864acfb6111fbe\": container with ID starting with 71a71fbd077ce90eb4407605b347850051482cfe3bb53f06ee864acfb6111fbe not found: ID does not exist" containerID="71a71fbd077ce90eb4407605b347850051482cfe3bb53f06ee864acfb6111fbe"
Jul 28 19:41:41.784749 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:41.784722    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71a71fbd077ce90eb4407605b347850051482cfe3bb53f06ee864acfb6111fbe"} err="failed to get container status \"71a71fbd077ce90eb4407605b347850051482cfe3bb53f06ee864acfb6111fbe\": rpc error: code = NotFound desc = could not find container \"71a71fbd077ce90eb4407605b347850051482cfe3bb53f06ee864acfb6111fbe\": container with ID starting with 71a71fbd077ce90eb4407605b347850051482cfe3bb53f06ee864acfb6111fbe not found: ID does not exist"
Jul 28 19:41:41.794642 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:41.794615    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-2k9mm"]
Jul 28 19:41:41.805817 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:41.800796    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-2k9mm"]
Jul 28 19:41:42.864149 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:42.864120    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c968f60c-bdbe-494f-a5c5-4ada24581c59" path="/var/lib/kubelet/pods/c968f60c-bdbe-494f-a5c5-4ada24581c59/volumes"
Jul 28 19:41:51.155801 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:51.155763    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:51.256318 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:51.256285    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:51.642964 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:41:51.642930    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:03.441461 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:03.441424    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:03.563248 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:03.563219    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:03.910269 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:03.910241    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5f898467cf-hvhlf"]
Jul 28 19:42:03.910596 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:03.910583    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c968f60c-bdbe-494f-a5c5-4ada24581c59" containerName="authorino"
Jul 28 19:42:03.910641 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:03.910597    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c968f60c-bdbe-494f-a5c5-4ada24581c59" containerName="authorino"
Jul 28 19:42:03.910675 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:03.910667    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c968f60c-bdbe-494f-a5c5-4ada24581c59" containerName="authorino"
Jul 28 19:42:03.913736 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:03.913717    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f898467cf-hvhlf"
Jul 28 19:42:03.918450 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:03.918424    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5f898467cf-hvhlf"]
Jul 28 19:42:04.033621 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:04.033598    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5f898467cf-hvhlf"]
Jul 28 19:42:04.033797 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:42:04.033778    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-jfb9x], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5f898467cf-hvhlf" podUID="95691189-892c-4b41-8950-835b6e05f889"
Jul 28 19:42:04.079003 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:04.078978    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfb9x\" (UniqueName: \"kubernetes.io/projected/95691189-892c-4b41-8950-835b6e05f889-kube-api-access-jfb9x\") pod \"authorino-5f898467cf-hvhlf\" (UID: \"95691189-892c-4b41-8950-835b6e05f889\") " pod="kuadrant-system/authorino-5f898467cf-hvhlf"
Jul 28 19:42:04.179773 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:04.179719    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfb9x\" (UniqueName: \"kubernetes.io/projected/95691189-892c-4b41-8950-835b6e05f889-kube-api-access-jfb9x\") pod \"authorino-5f898467cf-hvhlf\" (UID: \"95691189-892c-4b41-8950-835b6e05f889\") " pod="kuadrant-system/authorino-5f898467cf-hvhlf"
Jul 28 19:42:04.187905 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:04.187876    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfb9x\" (UniqueName: \"kubernetes.io/projected/95691189-892c-4b41-8950-835b6e05f889-kube-api-access-jfb9x\") pod \"authorino-5f898467cf-hvhlf\" (UID: \"95691189-892c-4b41-8950-835b6e05f889\") " pod="kuadrant-system/authorino-5f898467cf-hvhlf"
Jul 28 19:42:04.895081 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:04.895053    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f898467cf-hvhlf"
Jul 28 19:42:04.902081 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:04.902057    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f898467cf-hvhlf"
Jul 28 19:42:04.986263 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:04.986245    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfb9x\" (UniqueName: \"kubernetes.io/projected/95691189-892c-4b41-8950-835b6e05f889-kube-api-access-jfb9x\") pod \"95691189-892c-4b41-8950-835b6e05f889\" (UID: \"95691189-892c-4b41-8950-835b6e05f889\") "
Jul 28 19:42:04.988108 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:04.988084    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95691189-892c-4b41-8950-835b6e05f889-kube-api-access-jfb9x" (OuterVolumeSpecName: "kube-api-access-jfb9x") pod "95691189-892c-4b41-8950-835b6e05f889" (UID: "95691189-892c-4b41-8950-835b6e05f889"). InnerVolumeSpecName "kube-api-access-jfb9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:42:05.086792 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:05.086761    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfb9x\" (UniqueName: \"kubernetes.io/projected/95691189-892c-4b41-8950-835b6e05f889-kube-api-access-jfb9x\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:42:05.899453 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:05.899425    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f898467cf-hvhlf"
Jul 28 19:42:05.927558 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:05.927531    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5f898467cf-hvhlf"]
Jul 28 19:42:05.930718 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:05.930696    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5f898467cf-hvhlf"]
Jul 28 19:42:06.863339 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:06.863309    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95691189-892c-4b41-8950-835b6e05f889" path="/var/lib/kubelet/pods/95691189-892c-4b41-8950-835b6e05f889/volumes"
Jul 28 19:42:07.882950 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:07.882911    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-frxsh"]
Jul 28 19:42:07.883424 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:07.883151    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-frxsh" podUID="f63d08ab-939f-4062-a590-daf9bb061c06" containerName="authorino" containerID="cri-o://c11790f82627f0957a176d4e65031f349b103fd3620f57f9854040ee255cdf72" gracePeriod=30
Jul 28 19:42:08.129050 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:08.129030    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-frxsh"
Jul 28 19:42:08.211438 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:08.211382    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfb9g\" (UniqueName: \"kubernetes.io/projected/f63d08ab-939f-4062-a590-daf9bb061c06-kube-api-access-wfb9g\") pod \"f63d08ab-939f-4062-a590-daf9bb061c06\" (UID: \"f63d08ab-939f-4062-a590-daf9bb061c06\") "
Jul 28 19:42:08.213276 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:08.213251    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f63d08ab-939f-4062-a590-daf9bb061c06-kube-api-access-wfb9g" (OuterVolumeSpecName: "kube-api-access-wfb9g") pod "f63d08ab-939f-4062-a590-daf9bb061c06" (UID: "f63d08ab-939f-4062-a590-daf9bb061c06"). InnerVolumeSpecName "kube-api-access-wfb9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:42:08.312793 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:08.312761    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfb9g\" (UniqueName: \"kubernetes.io/projected/f63d08ab-939f-4062-a590-daf9bb061c06-kube-api-access-wfb9g\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:42:08.915022 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:08.914993    2572 generic.go:356] "Generic (PLEG): container finished" podID="f63d08ab-939f-4062-a590-daf9bb061c06" containerID="c11790f82627f0957a176d4e65031f349b103fd3620f57f9854040ee255cdf72" exitCode=0
Jul 28 19:42:08.915390 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:08.915037    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-frxsh"
Jul 28 19:42:08.915390 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:08.915064    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-frxsh" event={"ID":"f63d08ab-939f-4062-a590-daf9bb061c06","Type":"ContainerDied","Data":"c11790f82627f0957a176d4e65031f349b103fd3620f57f9854040ee255cdf72"}
Jul 28 19:42:08.915390 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:08.915120    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-frxsh" event={"ID":"f63d08ab-939f-4062-a590-daf9bb061c06","Type":"ContainerDied","Data":"df5aef8211a417fe5abe06a763ea4f4dbf4a78dfce0c9a777894d0b11ae30380"}
Jul 28 19:42:08.915390 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:08.915138    2572 scope.go:117] "RemoveContainer" containerID="c11790f82627f0957a176d4e65031f349b103fd3620f57f9854040ee255cdf72"
Jul 28 19:42:08.924041 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:08.924023    2572 scope.go:117] "RemoveContainer" containerID="c11790f82627f0957a176d4e65031f349b103fd3620f57f9854040ee255cdf72"
Jul 28 19:42:08.924326 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:42:08.924307    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c11790f82627f0957a176d4e65031f349b103fd3620f57f9854040ee255cdf72\": container with ID starting with c11790f82627f0957a176d4e65031f349b103fd3620f57f9854040ee255cdf72 not found: ID does not exist" containerID="c11790f82627f0957a176d4e65031f349b103fd3620f57f9854040ee255cdf72"
Jul 28 19:42:08.924379 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:08.924334    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c11790f82627f0957a176d4e65031f349b103fd3620f57f9854040ee255cdf72"} err="failed to get container status \"c11790f82627f0957a176d4e65031f349b103fd3620f57f9854040ee255cdf72\": rpc error: code = NotFound desc = could not find container \"c11790f82627f0957a176d4e65031f349b103fd3620f57f9854040ee255cdf72\": container with ID starting with c11790f82627f0957a176d4e65031f349b103fd3620f57f9854040ee255cdf72 not found: ID does not exist"
Jul 28 19:42:08.931104 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:08.931080    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-frxsh"]
Jul 28 19:42:08.936824 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:08.936804    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-frxsh"]
Jul 28 19:42:10.864418 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:10.864385    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f63d08ab-939f-4062-a590-daf9bb061c06" path="/var/lib/kubelet/pods/f63d08ab-939f-4062-a590-daf9bb061c06/volumes"
Jul 28 19:42:12.624956 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.624912    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:12.643738 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.643712    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f44d66b84-cqsfg"]
Jul 28 19:42:12.644103 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.644087    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f63d08ab-939f-4062-a590-daf9bb061c06" containerName="authorino"
Jul 28 19:42:12.644182 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.644106    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f63d08ab-939f-4062-a590-daf9bb061c06" containerName="authorino"
Jul 28 19:42:12.644235 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.644216    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f63d08ab-939f-4062-a590-daf9bb061c06" containerName="authorino"
Jul 28 19:42:12.648662 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.648640    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f44d66b84-cqsfg"
Jul 28 19:42:12.651418 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.651393    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 19:42:12.651539 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.651425    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 19:42:12.651539 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.651393    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6kzd6\""
Jul 28 19:42:12.657172 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.657141    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f44d66b84-cqsfg"]
Jul 28 19:42:12.696922 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.696896    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:12.743289 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.743268    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77rr5\" (UniqueName: \"kubernetes.io/projected/e0185583-a0ac-4bbf-b2a3-6672e147ec97-kube-api-access-77rr5\") pod \"maas-controller-5f44d66b84-cqsfg\" (UID: \"e0185583-a0ac-4bbf-b2a3-6672e147ec97\") " pod="opendatahub/maas-controller-5f44d66b84-cqsfg"
Jul 28 19:42:12.743397 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.743298    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e0185583-a0ac-4bbf-b2a3-6672e147ec97-webhook-cert\") pod \"maas-controller-5f44d66b84-cqsfg\" (UID: \"e0185583-a0ac-4bbf-b2a3-6672e147ec97\") " pod="opendatahub/maas-controller-5f44d66b84-cqsfg"
Jul 28 19:42:12.751298 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.751272    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:12.843054 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.843034    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:12.844194 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.844177    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77rr5\" (UniqueName: \"kubernetes.io/projected/e0185583-a0ac-4bbf-b2a3-6672e147ec97-kube-api-access-77rr5\") pod \"maas-controller-5f44d66b84-cqsfg\" (UID: \"e0185583-a0ac-4bbf-b2a3-6672e147ec97\") " pod="opendatahub/maas-controller-5f44d66b84-cqsfg"
Jul 28 19:42:12.844259 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.844206    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e0185583-a0ac-4bbf-b2a3-6672e147ec97-webhook-cert\") pod \"maas-controller-5f44d66b84-cqsfg\" (UID: \"e0185583-a0ac-4bbf-b2a3-6672e147ec97\") " pod="opendatahub/maas-controller-5f44d66b84-cqsfg"
Jul 28 19:42:12.846482 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.846465    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e0185583-a0ac-4bbf-b2a3-6672e147ec97-webhook-cert\") pod \"maas-controller-5f44d66b84-cqsfg\" (UID: \"e0185583-a0ac-4bbf-b2a3-6672e147ec97\") " pod="opendatahub/maas-controller-5f44d66b84-cqsfg"
Jul 28 19:42:12.851943 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.851920    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77rr5\" (UniqueName: \"kubernetes.io/projected/e0185583-a0ac-4bbf-b2a3-6672e147ec97-kube-api-access-77rr5\") pod \"maas-controller-5f44d66b84-cqsfg\" (UID: \"e0185583-a0ac-4bbf-b2a3-6672e147ec97\") " pod="opendatahub/maas-controller-5f44d66b84-cqsfg"
Jul 28 19:42:12.961486 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:12.961436    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f44d66b84-cqsfg"
Jul 28 19:42:13.084060 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:13.084030    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f44d66b84-cqsfg"]
Jul 28 19:42:13.085451 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:42:13.085426    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0185583_a0ac_4bbf_b2a3_6672e147ec97.slice/crio-755b06d6cf0802a6955251e69d825accc2d6b72adfb893927040b56029d27f12 WatchSource:0}: Error finding container 755b06d6cf0802a6955251e69d825accc2d6b72adfb893927040b56029d27f12: Status 404 returned error can't find the container with id 755b06d6cf0802a6955251e69d825accc2d6b72adfb893927040b56029d27f12
Jul 28 19:42:13.944732 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:13.944666    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f44d66b84-cqsfg" event={"ID":"e0185583-a0ac-4bbf-b2a3-6672e147ec97","Type":"ContainerStarted","Data":"755b06d6cf0802a6955251e69d825accc2d6b72adfb893927040b56029d27f12"}
Jul 28 19:42:15.460285 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:15.460247    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:15.559401 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:15.559371    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:15.956621 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:15.956541    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f44d66b84-cqsfg" event={"ID":"e0185583-a0ac-4bbf-b2a3-6672e147ec97","Type":"ContainerStarted","Data":"c6507c6263a5ca233b7154a5018f15974bac6397bf16b71dbec6433b040e3867"}
Jul 28 19:42:15.956621 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:15.956608    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f44d66b84-cqsfg"
Jul 28 19:42:15.978442 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:15.978400    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f44d66b84-cqsfg" podStartSLOduration=1.7651245009999998 podStartE2EDuration="3.978384829s" totalImagesPullingTime="2.213260328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:15.976883508 +0000 UTC m=+761.564546454" watchObservedRunningTime="2026-07-28 19:42:15.978384829 +0000 UTC m=+761.566047774"
Jul 28 19:42:26.963883 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:26.963854    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f44d66b84-cqsfg"
Jul 28 19:42:27.224798 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:27.224721    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f44d66b84-cqsfg"]
Jul 28 19:42:27.224939 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:27.224882    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5f44d66b84-cqsfg" podUID="e0185583-a0ac-4bbf-b2a3-6672e147ec97" containerName="manager" containerID="cri-o://c6507c6263a5ca233b7154a5018f15974bac6397bf16b71dbec6433b040e3867" gracePeriod=30
Jul 28 19:42:27.459586 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:27.459565    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f44d66b84-cqsfg"
Jul 28 19:42:27.566832 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:27.566812    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77rr5\" (UniqueName: \"kubernetes.io/projected/e0185583-a0ac-4bbf-b2a3-6672e147ec97-kube-api-access-77rr5\") pod \"e0185583-a0ac-4bbf-b2a3-6672e147ec97\" (UID: \"e0185583-a0ac-4bbf-b2a3-6672e147ec97\") "
Jul 28 19:42:27.566941 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:27.566888    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e0185583-a0ac-4bbf-b2a3-6672e147ec97-webhook-cert\") pod \"e0185583-a0ac-4bbf-b2a3-6672e147ec97\" (UID: \"e0185583-a0ac-4bbf-b2a3-6672e147ec97\") "
Jul 28 19:42:27.568895 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:27.568869    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0185583-a0ac-4bbf-b2a3-6672e147ec97-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e0185583-a0ac-4bbf-b2a3-6672e147ec97" (UID: "e0185583-a0ac-4bbf-b2a3-6672e147ec97"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:42:27.569002 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:27.568918    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0185583-a0ac-4bbf-b2a3-6672e147ec97-kube-api-access-77rr5" (OuterVolumeSpecName: "kube-api-access-77rr5") pod "e0185583-a0ac-4bbf-b2a3-6672e147ec97" (UID: "e0185583-a0ac-4bbf-b2a3-6672e147ec97"). InnerVolumeSpecName "kube-api-access-77rr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:42:27.667573 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:27.667542    2572 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e0185583-a0ac-4bbf-b2a3-6672e147ec97-webhook-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:42:27.667573 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:27.667572    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77rr5\" (UniqueName: \"kubernetes.io/projected/e0185583-a0ac-4bbf-b2a3-6672e147ec97-kube-api-access-77rr5\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:42:28.018375 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:28.018309    2572 generic.go:356] "Generic (PLEG): container finished" podID="e0185583-a0ac-4bbf-b2a3-6672e147ec97" containerID="c6507c6263a5ca233b7154a5018f15974bac6397bf16b71dbec6433b040e3867" exitCode=0
Jul 28 19:42:28.018375 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:28.018360    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f44d66b84-cqsfg"
Jul 28 19:42:28.018843 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:28.018406    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f44d66b84-cqsfg" event={"ID":"e0185583-a0ac-4bbf-b2a3-6672e147ec97","Type":"ContainerDied","Data":"c6507c6263a5ca233b7154a5018f15974bac6397bf16b71dbec6433b040e3867"}
Jul 28 19:42:28.018843 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:28.018478    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f44d66b84-cqsfg" event={"ID":"e0185583-a0ac-4bbf-b2a3-6672e147ec97","Type":"ContainerDied","Data":"755b06d6cf0802a6955251e69d825accc2d6b72adfb893927040b56029d27f12"}
Jul 28 19:42:28.018843 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:28.018517    2572 scope.go:117] "RemoveContainer" containerID="c6507c6263a5ca233b7154a5018f15974bac6397bf16b71dbec6433b040e3867"
Jul 28 19:42:28.028492 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:28.028474    2572 scope.go:117] "RemoveContainer" containerID="c6507c6263a5ca233b7154a5018f15974bac6397bf16b71dbec6433b040e3867"
Jul 28 19:42:28.028778 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:42:28.028759    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6507c6263a5ca233b7154a5018f15974bac6397bf16b71dbec6433b040e3867\": container with ID starting with c6507c6263a5ca233b7154a5018f15974bac6397bf16b71dbec6433b040e3867 not found: ID does not exist" containerID="c6507c6263a5ca233b7154a5018f15974bac6397bf16b71dbec6433b040e3867"
Jul 28 19:42:28.028841 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:28.028792    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6507c6263a5ca233b7154a5018f15974bac6397bf16b71dbec6433b040e3867"} err="failed to get container status \"c6507c6263a5ca233b7154a5018f15974bac6397bf16b71dbec6433b040e3867\": rpc error: code = NotFound desc = could not find container \"c6507c6263a5ca233b7154a5018f15974bac6397bf16b71dbec6433b040e3867\": container with ID starting with c6507c6263a5ca233b7154a5018f15974bac6397bf16b71dbec6433b040e3867 not found: ID does not exist"
Jul 28 19:42:28.039692 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:28.039667    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f44d66b84-cqsfg"]
Jul 28 19:42:28.042692 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:28.042668    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f44d66b84-cqsfg"]
Jul 28 19:42:28.863979 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:28.863944    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0185583-a0ac-4bbf-b2a3-6672e147ec97" path="/var/lib/kubelet/pods/e0185583-a0ac-4bbf-b2a3-6672e147ec97/volumes"
Jul 28 19:42:48.119347 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:48.119312    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:48.212502 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:48.212477    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:48.599565 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:48.599536    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:48.805429 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:48.805406    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:48.915746 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:48.915681    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m"]
Jul 28 19:42:48.916008 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:48.915997    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0185583-a0ac-4bbf-b2a3-6672e147ec97" containerName="manager"
Jul 28 19:42:48.916049 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:48.916009    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0185583-a0ac-4bbf-b2a3-6672e147ec97" containerName="manager"
Jul 28 19:42:48.916118 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:48.916082    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0185583-a0ac-4bbf-b2a3-6672e147ec97" containerName="manager"
Jul 28 19:42:48.921525 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:48.921507    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m"
Jul 28 19:42:48.924875 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:48.924855    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:42:48.924985 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:48.924874    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 28 19:42:48.924985 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:48.924861    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 19:42:48.924985 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:48.924910    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g5qmq\""
Jul 28 19:42:48.930031 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:48.930005    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m"]
Jul 28 19:42:49.016940 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.016917    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xq7f\" (UniqueName: \"kubernetes.io/projected/1c8f71f2-d0fb-433e-a181-aa55887b303f-kube-api-access-4xq7f\") pod \"maas-api-6f48d844d4-g6v4m\" (UID: \"1c8f71f2-d0fb-433e-a181-aa55887b303f\") " pod="odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m"
Jul 28 19:42:49.017039 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.016953    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c8f71f2-d0fb-433e-a181-aa55887b303f-openshift-service-ca\") pod \"maas-api-6f48d844d4-g6v4m\" (UID: \"1c8f71f2-d0fb-433e-a181-aa55887b303f\") " pod="odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m"
Jul 28 19:42:49.017039 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.017005    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c8f71f2-d0fb-433e-a181-aa55887b303f-maas-api-tls\") pod \"maas-api-6f48d844d4-g6v4m\" (UID: \"1c8f71f2-d0fb-433e-a181-aa55887b303f\") " pod="odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m"
Jul 28 19:42:49.117395 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.117367    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c8f71f2-d0fb-433e-a181-aa55887b303f-maas-api-tls\") pod \"maas-api-6f48d844d4-g6v4m\" (UID: \"1c8f71f2-d0fb-433e-a181-aa55887b303f\") " pod="odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m"
Jul 28 19:42:49.117515 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.117411    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xq7f\" (UniqueName: \"kubernetes.io/projected/1c8f71f2-d0fb-433e-a181-aa55887b303f-kube-api-access-4xq7f\") pod \"maas-api-6f48d844d4-g6v4m\" (UID: \"1c8f71f2-d0fb-433e-a181-aa55887b303f\") " pod="odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m"
Jul 28 19:42:49.117581 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.117562    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c8f71f2-d0fb-433e-a181-aa55887b303f-openshift-service-ca\") pod \"maas-api-6f48d844d4-g6v4m\" (UID: \"1c8f71f2-d0fb-433e-a181-aa55887b303f\") " pod="odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m"
Jul 28 19:42:49.118117 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.118097    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c8f71f2-d0fb-433e-a181-aa55887b303f-openshift-service-ca\") pod \"maas-api-6f48d844d4-g6v4m\" (UID: \"1c8f71f2-d0fb-433e-a181-aa55887b303f\") " pod="odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m"
Jul 28 19:42:49.119904 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.119881    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c8f71f2-d0fb-433e-a181-aa55887b303f-maas-api-tls\") pod \"maas-api-6f48d844d4-g6v4m\" (UID: \"1c8f71f2-d0fb-433e-a181-aa55887b303f\") " pod="odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m"
Jul 28 19:42:49.124871 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.124852    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xq7f\" (UniqueName: \"kubernetes.io/projected/1c8f71f2-d0fb-433e-a181-aa55887b303f-kube-api-access-4xq7f\") pod \"maas-api-6f48d844d4-g6v4m\" (UID: \"1c8f71f2-d0fb-433e-a181-aa55887b303f\") " pod="odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m"
Jul 28 19:42:49.232114 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.232032    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m"
Jul 28 19:42:49.273271 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.273250    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:49.367969 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.367942    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m"]
Jul 28 19:42:49.369906 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:42:49.369880    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c8f71f2_d0fb_433e_a181_aa55887b303f.slice/crio-679b234d104001b41011daf0a42b1f9dec8dc139ba1b51a76e04de4c1f3681e1 WatchSource:0}: Error finding container 679b234d104001b41011daf0a42b1f9dec8dc139ba1b51a76e04de4c1f3681e1: Status 404 returned error can't find the container with id 679b234d104001b41011daf0a42b1f9dec8dc139ba1b51a76e04de4c1f3681e1
Jul 28 19:42:49.375112 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.375061    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:49.379545 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.379524    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-gqmd8"]
Jul 28 19:42:49.383821 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.383804    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-gqmd8"
Jul 28 19:42:49.386340 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.386316    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 28 19:42:49.386532 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.386514    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-pl8rm\""
Jul 28 19:42:49.392349 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.392332    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-gqmd8"]
Jul 28 19:42:49.420778 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.420759    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-gqmd8\" (UID: \"5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-gqmd8"
Jul 28 19:42:49.420882 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.420812    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jccp\" (UniqueName: \"kubernetes.io/projected/5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210-kube-api-access-9jccp\") pod \"payload-pre-processing-687dbb6cf6-gqmd8\" (UID: \"5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-gqmd8"
Jul 28 19:42:49.522024 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.521975    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-gqmd8\" (UID: \"5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-gqmd8"
Jul 28 19:42:49.522024 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.522011    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jccp\" (UniqueName: \"kubernetes.io/projected/5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210-kube-api-access-9jccp\") pod \"payload-pre-processing-687dbb6cf6-gqmd8\" (UID: \"5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-gqmd8"
Jul 28 19:42:49.522612 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.522595    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-gqmd8\" (UID: \"5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-gqmd8"
Jul 28 19:42:49.529330 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.529292    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jccp\" (UniqueName: \"kubernetes.io/projected/5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210-kube-api-access-9jccp\") pod \"payload-pre-processing-687dbb6cf6-gqmd8\" (UID: \"5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-gqmd8"
Jul 28 19:42:49.695201 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:49.695162    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-gqmd8"
Jul 28 19:42:50.024542 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:50.024517    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-gqmd8"]
Jul 28 19:42:50.026954 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:42:50.026926    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b1421c8_eed0_42d9_ac4a_6bb7a9fbb210.slice/crio-d1c377a89b76d315e5f187bd4a3d03cb4e1c32abbbb924c27e2b2293c52353ce WatchSource:0}: Error finding container d1c377a89b76d315e5f187bd4a3d03cb4e1c32abbbb924c27e2b2293c52353ce: Status 404 returned error can't find the container with id d1c377a89b76d315e5f187bd4a3d03cb4e1c32abbbb924c27e2b2293c52353ce
Jul 28 19:42:50.130159 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:50.130119    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m" event={"ID":"1c8f71f2-d0fb-433e-a181-aa55887b303f","Type":"ContainerStarted","Data":"679b234d104001b41011daf0a42b1f9dec8dc139ba1b51a76e04de4c1f3681e1"}
Jul 28 19:42:50.131512 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:50.131476    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-gqmd8" event={"ID":"5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210","Type":"ContainerStarted","Data":"d1c377a89b76d315e5f187bd4a3d03cb4e1c32abbbb924c27e2b2293c52353ce"}
Jul 28 19:42:52.146892 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:52.145979    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m" event={"ID":"1c8f71f2-d0fb-433e-a181-aa55887b303f","Type":"ContainerStarted","Data":"6e8a093a7b9240d4af56f6ad2e09eb5ae7a1d7316e1fa904273e003094e0cbc3"}
Jul 28 19:42:52.146892 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:52.146824    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m"
Jul 28 19:42:52.149014 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:52.148983    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-gqmd8" event={"ID":"5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210","Type":"ContainerStarted","Data":"6ab7873e59fabd2d0749ce60afccd49caf5ca82be7eb608d1276cbdfab4a427c"}
Jul 28 19:42:52.149489 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:52.149465    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-gqmd8"
Jul 28 19:42:52.164697 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:52.164650    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m" podStartSLOduration=2.207045145 podStartE2EDuration="4.164632953s" totalImagesPullingTime="1.957587808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:52.162899143 +0000 UTC m=+797.750562089" watchObservedRunningTime="2026-07-28 19:42:52.164632953 +0000 UTC m=+797.752295905"
Jul 28 19:42:52.179054 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:52.179021    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-gqmd8" podStartSLOduration=1.228886928 podStartE2EDuration="3.179007884s" totalImagesPullingTime="1.950120956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:52.177792436 +0000 UTC m=+797.765455398" watchObservedRunningTime="2026-07-28 19:42:52.179007884 +0000 UTC m=+797.766670829"
Jul 28 19:42:57.187440 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:57.187408    2572 generic.go:356] "Generic (PLEG): container finished" podID="36b15eb7-edfa-4803-8270-332cafadae6c" containerID="b0148a088fc5dad74d3fca7c59ed746b4ebb273bbadf7d5bccdd18eb9cad6742" exitCode=0
Jul 28 19:42:57.187865 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:57.187449    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9" event={"ID":"36b15eb7-edfa-4803-8270-332cafadae6c","Type":"ContainerDied","Data":"b0148a088fc5dad74d3fca7c59ed746b4ebb273bbadf7d5bccdd18eb9cad6742"}
Jul 28 19:42:57.187865 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:57.187765    2572 scope.go:117] "RemoveContainer" containerID="b0148a088fc5dad74d3fca7c59ed746b4ebb273bbadf7d5bccdd18eb9cad6742"
Jul 28 19:42:57.188689 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:57.188661    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:42:57.188742 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:57.188726    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:42:57.188777 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:57.188754    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:42:58.193833 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:58.193803    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9" event={"ID":"36b15eb7-edfa-4803-8270-332cafadae6c","Type":"ContainerStarted","Data":"5b6e523d33f308247d3424f2f063ae02efabd7621bb2fdee975207500c9d11f3"}
Jul 28 19:42:59.157985 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:59.157952    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:42:59.158372 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:59.158287    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-c5xr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 28 19:42:59.158563 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:59.158540    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9" podUID="36b15eb7-edfa-4803-8270-332cafadae6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 28 19:42:59.170129 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:42:59.170109    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6f48d844d4-g6v4m"
Jul 28 19:43:00.158203 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:00.158165    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-c5xr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 28 19:43:00.158695 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:00.158226    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9" podUID="36b15eb7-edfa-4803-8270-332cafadae6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 28 19:43:00.225578 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:00.225544    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:00.324063 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:00.324034    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:01.157690 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:01.157655    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-c5xr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 28 19:43:01.158335 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:01.157957    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9" podUID="36b15eb7-edfa-4803-8270-332cafadae6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 28 19:43:02.161796 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:02.161767    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:43:02.162187 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:02.162019    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:43:02.162662 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:02.162646    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c5xr9"
Jul 28 19:43:04.164450 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:04.164408    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-gqmd8" podUID="5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.48:9004: connect: connection refused"
Jul 28 19:43:05.029209 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:05.029184    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:06.098712 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:06.098675    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:14.163097 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:14.163047    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-gqmd8"
Jul 28 19:43:21.572896 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.572777    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"]
Jul 28 19:43:21.576898 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.576869    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.580573 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.580533    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-csdqr\""
Jul 28 19:43:21.580573 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.580540    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 19:43:21.580573 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.580571    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 19:43:21.580884 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.580778    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 28 19:43:21.586316 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.586276    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"]
Jul 28 19:43:21.619431 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.619407    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:21.691328 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.691302    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96c06d46-1e9d-46ce-a698-d8d7bda9cb57-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vdk4\" (UID: \"96c06d46-1e9d-46ce-a698-d8d7bda9cb57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.691445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.691345    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96c06d46-1e9d-46ce-a698-d8d7bda9cb57-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vdk4\" (UID: \"96c06d46-1e9d-46ce-a698-d8d7bda9cb57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.691445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.691368    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96c06d46-1e9d-46ce-a698-d8d7bda9cb57-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vdk4\" (UID: \"96c06d46-1e9d-46ce-a698-d8d7bda9cb57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.691445 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.691394    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96c06d46-1e9d-46ce-a698-d8d7bda9cb57-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vdk4\" (UID: \"96c06d46-1e9d-46ce-a698-d8d7bda9cb57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.691561 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.691468    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96c06d46-1e9d-46ce-a698-d8d7bda9cb57-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vdk4\" (UID: \"96c06d46-1e9d-46ce-a698-d8d7bda9cb57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.691561 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.691515    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4njxg\" (UniqueName: \"kubernetes.io/projected/96c06d46-1e9d-46ce-a698-d8d7bda9cb57-kube-api-access-4njxg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vdk4\" (UID: \"96c06d46-1e9d-46ce-a698-d8d7bda9cb57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.792668 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.792641    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96c06d46-1e9d-46ce-a698-d8d7bda9cb57-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vdk4\" (UID: \"96c06d46-1e9d-46ce-a698-d8d7bda9cb57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.792668 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.792671    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96c06d46-1e9d-46ce-a698-d8d7bda9cb57-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vdk4\" (UID: \"96c06d46-1e9d-46ce-a698-d8d7bda9cb57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.792855 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.792689    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96c06d46-1e9d-46ce-a698-d8d7bda9cb57-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vdk4\" (UID: \"96c06d46-1e9d-46ce-a698-d8d7bda9cb57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.792855 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.792717    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96c06d46-1e9d-46ce-a698-d8d7bda9cb57-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vdk4\" (UID: \"96c06d46-1e9d-46ce-a698-d8d7bda9cb57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.792855 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.792764    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4njxg\" (UniqueName: \"kubernetes.io/projected/96c06d46-1e9d-46ce-a698-d8d7bda9cb57-kube-api-access-4njxg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vdk4\" (UID: \"96c06d46-1e9d-46ce-a698-d8d7bda9cb57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.792855 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.792817    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96c06d46-1e9d-46ce-a698-d8d7bda9cb57-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vdk4\" (UID: \"96c06d46-1e9d-46ce-a698-d8d7bda9cb57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.793168 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.793141    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96c06d46-1e9d-46ce-a698-d8d7bda9cb57-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vdk4\" (UID: \"96c06d46-1e9d-46ce-a698-d8d7bda9cb57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.793250 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.793170    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96c06d46-1e9d-46ce-a698-d8d7bda9cb57-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vdk4\" (UID: \"96c06d46-1e9d-46ce-a698-d8d7bda9cb57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.793250 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.793217    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96c06d46-1e9d-46ce-a698-d8d7bda9cb57-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vdk4\" (UID: \"96c06d46-1e9d-46ce-a698-d8d7bda9cb57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.794918 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.794887    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96c06d46-1e9d-46ce-a698-d8d7bda9cb57-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vdk4\" (UID: \"96c06d46-1e9d-46ce-a698-d8d7bda9cb57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.795268 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.795249    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96c06d46-1e9d-46ce-a698-d8d7bda9cb57-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vdk4\" (UID: \"96c06d46-1e9d-46ce-a698-d8d7bda9cb57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.799992 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.799973    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4njxg\" (UniqueName: \"kubernetes.io/projected/96c06d46-1e9d-46ce-a698-d8d7bda9cb57-kube-api-access-4njxg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vdk4\" (UID: \"96c06d46-1e9d-46ce-a698-d8d7bda9cb57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:21.892733 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:21.892682    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:22.017613 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:22.017588    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"]
Jul 28 19:43:22.019012 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:43:22.018987    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96c06d46_1e9d_46ce_a698_d8d7bda9cb57.slice/crio-88be02fb104fd79e12e766e6d23368bfc219097a6d7ea8da93c959b4e80e4a45 WatchSource:0}: Error finding container 88be02fb104fd79e12e766e6d23368bfc219097a6d7ea8da93c959b4e80e4a45: Status 404 returned error can't find the container with id 88be02fb104fd79e12e766e6d23368bfc219097a6d7ea8da93c959b4e80e4a45
Jul 28 19:43:22.324559 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:22.324528    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4" event={"ID":"96c06d46-1e9d-46ce-a698-d8d7bda9cb57","Type":"ContainerStarted","Data":"88be02fb104fd79e12e766e6d23368bfc219097a6d7ea8da93c959b4e80e4a45"}
Jul 28 19:43:23.331149 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:23.331117    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4" event={"ID":"96c06d46-1e9d-46ce-a698-d8d7bda9cb57","Type":"ContainerStarted","Data":"5d166e626c7b980872e39e9e3060892bb4cb2e514fa5f80b9fa34d9a11fe1501"}
Jul 28 19:43:23.349816 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:23.349765    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4" podStartSLOduration=1.369238621 podStartE2EDuration="2.349750598s" totalImagesPullingTime="980.511977ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:43:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:43:23.347498345 +0000 UTC m=+828.935161278" watchObservedRunningTime="2026-07-28 19:43:23.349750598 +0000 UTC m=+828.937413543"
Jul 28 19:43:31.893721 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:31.893667    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:31.894387 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:31.893746    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:31.905809 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:31.905782    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:32.391625 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:32.391599    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vdk4"
Jul 28 19:43:47.220101 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:43:47.220040    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:10.180415 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:10.180379    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:10.234995 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:10.234964    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:33.881056 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:33.880974    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:34.775858 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:34.775831    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 19:44:34.779454 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:34.779430    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 19:44:39.670972 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.670927    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"]
Jul 28 19:44:39.675327 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.675303    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.677765 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.677738    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 28 19:44:39.683147 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.683115    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"]
Jul 28 19:44:39.712444 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.712413    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:39.792398 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.792375    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tnxs2\" (UID: \"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.792542 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.792423    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdrrs\" (UniqueName: \"kubernetes.io/projected/3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91-kube-api-access-hdrrs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tnxs2\" (UID: \"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.792542 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.792458    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tnxs2\" (UID: \"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.792542 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.792485    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tnxs2\" (UID: \"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.792542 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.792517    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tnxs2\" (UID: \"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.792713 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.792586    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tnxs2\" (UID: \"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.893301 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.893268    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tnxs2\" (UID: \"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.893433 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.893332    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tnxs2\" (UID: \"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.893433 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.893374    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdrrs\" (UniqueName: \"kubernetes.io/projected/3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91-kube-api-access-hdrrs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tnxs2\" (UID: \"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.893433 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.893407    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tnxs2\" (UID: \"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.893587 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.893433    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tnxs2\" (UID: \"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.893587 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.893469    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tnxs2\" (UID: \"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.893795 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.893769    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tnxs2\" (UID: \"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.893869 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.893800    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tnxs2\" (UID: \"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.893869 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.893837    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tnxs2\" (UID: \"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.895656 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.895635    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tnxs2\" (UID: \"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.895828 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.895811    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tnxs2\" (UID: \"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.900477 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.900453    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdrrs\" (UniqueName: \"kubernetes.io/projected/3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91-kube-api-access-hdrrs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tnxs2\" (UID: \"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:39.988767 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:39.988702    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:40.116294 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:40.116270    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"]
Jul 28 19:44:40.118022 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:44:40.117976    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c069a2c_ac9e_4ec9_b2d4_5b43ed400b91.slice/crio-3309b8bbda85282a66b3effc31c602edae8dccca576a612e7c184e6abfb1f1b2 WatchSource:0}: Error finding container 3309b8bbda85282a66b3effc31c602edae8dccca576a612e7c184e6abfb1f1b2: Status 404 returned error can't find the container with id 3309b8bbda85282a66b3effc31c602edae8dccca576a612e7c184e6abfb1f1b2
Jul 28 19:44:40.738381 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:40.738341    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2" event={"ID":"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91","Type":"ContainerStarted","Data":"568c34231eb3c0cc05a1aed18e3e54f1692cc53b4872b8878bb8615ee36305ed"}
Jul 28 19:44:40.738744 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:40.738407    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2" event={"ID":"3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91","Type":"ContainerStarted","Data":"3309b8bbda85282a66b3effc31c602edae8dccca576a612e7c184e6abfb1f1b2"}
Jul 28 19:44:40.755738 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:40.755653    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2" podStartSLOduration=1.382275261 podStartE2EDuration="1.755638362s" totalImagesPullingTime="373.363101ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:44:40.753767472 +0000 UTC m=+906.341430437" watchObservedRunningTime="2026-07-28 19:44:40.755638362 +0000 UTC m=+906.343301307"
Jul 28 19:44:43.182790 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.182746    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"]
Jul 28 19:44:43.187246 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.187221    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.189849 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.189815    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 28 19:44:43.194251 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.194224    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"]
Jul 28 19:44:43.211684 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.211651    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:43.319730 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.319708    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81e1b755-4c47-45fd-bf53-d8550103c3b8-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx\" (UID: \"81e1b755-4c47-45fd-bf53-d8550103c3b8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.319854 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.319750    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81e1b755-4c47-45fd-bf53-d8550103c3b8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx\" (UID: \"81e1b755-4c47-45fd-bf53-d8550103c3b8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.319854 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.319769    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81e1b755-4c47-45fd-bf53-d8550103c3b8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx\" (UID: \"81e1b755-4c47-45fd-bf53-d8550103c3b8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.319971 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.319893    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81e1b755-4c47-45fd-bf53-d8550103c3b8-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx\" (UID: \"81e1b755-4c47-45fd-bf53-d8550103c3b8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.319971 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.319932    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx59d\" (UniqueName: \"kubernetes.io/projected/81e1b755-4c47-45fd-bf53-d8550103c3b8-kube-api-access-jx59d\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx\" (UID: \"81e1b755-4c47-45fd-bf53-d8550103c3b8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.320111 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.320001    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81e1b755-4c47-45fd-bf53-d8550103c3b8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx\" (UID: \"81e1b755-4c47-45fd-bf53-d8550103c3b8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.421048 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.421016    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81e1b755-4c47-45fd-bf53-d8550103c3b8-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx\" (UID: \"81e1b755-4c47-45fd-bf53-d8550103c3b8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.421048 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.421055    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx59d\" (UniqueName: \"kubernetes.io/projected/81e1b755-4c47-45fd-bf53-d8550103c3b8-kube-api-access-jx59d\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx\" (UID: \"81e1b755-4c47-45fd-bf53-d8550103c3b8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.421220 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.421164    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81e1b755-4c47-45fd-bf53-d8550103c3b8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx\" (UID: \"81e1b755-4c47-45fd-bf53-d8550103c3b8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.421270 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.421252    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81e1b755-4c47-45fd-bf53-d8550103c3b8-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx\" (UID: \"81e1b755-4c47-45fd-bf53-d8550103c3b8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.421345 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.421329    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81e1b755-4c47-45fd-bf53-d8550103c3b8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx\" (UID: \"81e1b755-4c47-45fd-bf53-d8550103c3b8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.421392 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.421362    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81e1b755-4c47-45fd-bf53-d8550103c3b8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx\" (UID: \"81e1b755-4c47-45fd-bf53-d8550103c3b8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.421557 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.421528    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81e1b755-4c47-45fd-bf53-d8550103c3b8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx\" (UID: \"81e1b755-4c47-45fd-bf53-d8550103c3b8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.421686 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.421645    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81e1b755-4c47-45fd-bf53-d8550103c3b8-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx\" (UID: \"81e1b755-4c47-45fd-bf53-d8550103c3b8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.421686 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.421663    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81e1b755-4c47-45fd-bf53-d8550103c3b8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx\" (UID: \"81e1b755-4c47-45fd-bf53-d8550103c3b8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.423532 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.423509    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81e1b755-4c47-45fd-bf53-d8550103c3b8-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx\" (UID: \"81e1b755-4c47-45fd-bf53-d8550103c3b8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.423819 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.423800    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81e1b755-4c47-45fd-bf53-d8550103c3b8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx\" (UID: \"81e1b755-4c47-45fd-bf53-d8550103c3b8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.428294 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.428270    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx59d\" (UniqueName: \"kubernetes.io/projected/81e1b755-4c47-45fd-bf53-d8550103c3b8-kube-api-access-jx59d\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx\" (UID: \"81e1b755-4c47-45fd-bf53-d8550103c3b8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.500026 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.499968    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:43.628139 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.628108    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"]
Jul 28 19:44:43.629550 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:44:43.629519    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81e1b755_4c47_45fd_bf53_d8550103c3b8.slice/crio-9bc051f434efdfa347b80a2d506938b75d71412ab4059b92a54f3068d039a558 WatchSource:0}: Error finding container 9bc051f434efdfa347b80a2d506938b75d71412ab4059b92a54f3068d039a558: Status 404 returned error can't find the container with id 9bc051f434efdfa347b80a2d506938b75d71412ab4059b92a54f3068d039a558
Jul 28 19:44:43.755596 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:43.755538    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx" event={"ID":"81e1b755-4c47-45fd-bf53-d8550103c3b8","Type":"ContainerStarted","Data":"9bc051f434efdfa347b80a2d506938b75d71412ab4059b92a54f3068d039a558"}
Jul 28 19:44:44.762562 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:44.762530    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx" event={"ID":"81e1b755-4c47-45fd-bf53-d8550103c3b8","Type":"ContainerStarted","Data":"e9c2238cc67b4028ee923ca964d6c5e48fe9c15eb1b198a65cdd913e5ee3c399"}
Jul 28 19:44:44.788735 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:44.788692    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx" podStartSLOduration=1.500965587 podStartE2EDuration="1.788677085s" totalImagesPullingTime="287.711498ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:44:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:44:44.786794006 +0000 UTC m=+910.374456952" watchObservedRunningTime="2026-07-28 19:44:44.788677085 +0000 UTC m=+910.376340029"
Jul 28 19:44:49.988813 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:49.988778    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:49.989316 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:49.988947    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:50.001190 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:50.001166    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:50.806911 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:50.806883    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tnxs2"
Jul 28 19:44:53.500171 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:53.500132    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:53.500171 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:53.500185    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:53.512275 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:53.512246    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:44:53.824714 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:44:53.824693    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx"
Jul 28 19:45:59.544706 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:45:59.544620    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:46:00.752819 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:46:00.752782    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:46:26.712792 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:46:26.712756    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:46:46.692983 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:46:46.692940    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:46:47.905673 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:46:47.905629    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:09.272086 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:47:09.272047    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:09.372259 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:47:09.372224    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:05.131979 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:48:05.131944    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:06.347503 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:48:06.347463    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:20.503896 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:48:20.503847    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:54.547627 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:48:54.547597    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:54.647672 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:48:54.647646    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:54.752336 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:48:54.752312    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:00.358241 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:49:00.358201    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:00.462902 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:49:00.462849    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:00.570647 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:49:00.570617    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:00.713777 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:49:00.713684    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:06.267173 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:49:06.267127    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:06.336031 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:49:06.336001    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:34.809810 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:49:34.809782    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 19:49:34.814590 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:49:34.814566    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 19:50:41.703272 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:50:41.703180    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:02.619810 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:52:02.619681    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:05.612751 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:52:05.612720    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:17.959313 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:53:17.959265    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:34.845770 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:54:34.845739    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 19:54:34.850171 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:54:34.850150    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 19:55:53.887832 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:55:53.887789    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:53.974662 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:55:53.974625    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:07.195180 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.195142    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"]
Jul 28 19:57:07.198679 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.198661    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.201404 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.201373    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-6e7e0dcd-kserve-self-signed-certs\""
Jul 28 19:57:07.216722 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.216695    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"]
Jul 28 19:57:07.228978 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.228950    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:07.288347 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.288323    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-home\") pod \"e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") " pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.288444 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.288365    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-tmp-dir\") pod \"e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") " pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.288444 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.288428    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-model-cache\") pod \"e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") " pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.288519 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.288452    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-tls-certs\") pod \"e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") " pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.288519 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.288512    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78cpw\" (UniqueName: \"kubernetes.io/projected/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-kube-api-access-78cpw\") pod \"e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") " pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.288582 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.288528    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-dshm\") pod \"e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") " pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.389816 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.389729    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-model-cache\") pod \"e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") " pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.389931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.389844    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-tls-certs\") pod \"e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") " pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.389931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.389900    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78cpw\" (UniqueName: \"kubernetes.io/projected/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-kube-api-access-78cpw\") pod \"e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") " pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.389931 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.389918    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-dshm\") pod \"e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") " pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.390033 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.389951    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-home\") pod \"e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") " pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.390033 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.389984    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-tmp-dir\") pod \"e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") " pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.390131 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.390109    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-model-cache\") pod \"e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") " pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.390332 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.390307    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-home\") pod \"e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") " pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.390440 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.390387    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-tmp-dir\") pod \"e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") " pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.391991 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.391974    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-dshm\") pod \"e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") " pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.392323 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.392306    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-tls-certs\") pod \"e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") " pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.396997 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.396976    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78cpw\" (UniqueName: \"kubernetes.io/projected/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-kube-api-access-78cpw\") pod \"e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") " pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.516205 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.516148    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:07.646333 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.646308    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"]
Jul 28 19:57:07.648144 ip-10-0-138-37 kubenswrapper[2572]: W0728 19:57:07.648121    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podffbf99bc_98ef_46b5_9be9_7c9ec4b90ec5.slice/crio-aa6b8ce07a0b8af0480705233024f7d0270557f02c44de4be4f982fd50595cda WatchSource:0}: Error finding container aa6b8ce07a0b8af0480705233024f7d0270557f02c44de4be4f982fd50595cda: Status 404 returned error can't find the container with id aa6b8ce07a0b8af0480705233024f7d0270557f02c44de4be4f982fd50595cda
Jul 28 19:57:07.788251 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.788180    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr" event={"ID":"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5","Type":"ContainerStarted","Data":"1fcf11d5f36e225c48bb9c7699106077298377f934f721b03692b64e09c98dcf"}
Jul 28 19:57:07.788251 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.788221    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr" event={"ID":"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5","Type":"ContainerStarted","Data":"aa6b8ce07a0b8af0480705233024f7d0270557f02c44de4be4f982fd50595cda"}
Jul 28 19:57:07.806454 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:07.806411    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr" podStartSLOduration=0.806396608 podStartE2EDuration="806.396608ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:57:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:57:07.804741885 +0000 UTC m=+1653.392404829" watchObservedRunningTime="2026-07-28 19:57:07.806396608 +0000 UTC m=+1653.394059553"
Jul 28 19:57:17.517172 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:17.517133    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:17.517688 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:17.517190    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:17.529221 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:17.529192    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:17.860294 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:17.860267    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:57:18.948218 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:18.948188    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:53.667961 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:53.667915    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:53.767811 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:53.767778    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:58.284940 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:58.284853    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"]
Jul 28 19:57:58.313226 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:58.313199    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:59.803372 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:57:59.803339    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:06.475540 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:06.475484    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:13.303864 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.303792    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr" podUID="ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5" containerName="main" containerID="cri-o://1fcf11d5f36e225c48bb9c7699106077298377f934f721b03692b64e09c98dcf" gracePeriod=45
Jul 28 19:58:13.558789 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.558768    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:58:13.620083 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.620048    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78cpw\" (UniqueName: \"kubernetes.io/projected/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-kube-api-access-78cpw\") pod \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") "
Jul 28 19:58:13.620187 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.620107    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-tls-certs\") pod \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") "
Jul 28 19:58:13.620187 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.620157    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-home\") pod \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") "
Jul 28 19:58:13.620187 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.620174    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-model-cache\") pod \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") "
Jul 28 19:58:13.620354 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.620197    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-dshm\") pod \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") "
Jul 28 19:58:13.620354 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.620225    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-tmp-dir\") pod \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\" (UID: \"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5\") "
Jul 28 19:58:13.620494 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.620444    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-home" (OuterVolumeSpecName: "home") pod "ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5" (UID: "ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:58:13.620558 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.620471    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-model-cache" (OuterVolumeSpecName: "model-cache") pod "ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5" (UID: "ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:58:13.620558 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.620512    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5" (UID: "ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:58:13.622141 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.622116    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-dshm" (OuterVolumeSpecName: "dshm") pod "ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5" (UID: "ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:58:13.622235 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.622187    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5" (UID: "ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:58:13.622235 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.622226    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-kube-api-access-78cpw" (OuterVolumeSpecName: "kube-api-access-78cpw") pod "ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5" (UID: "ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5"). InnerVolumeSpecName "kube-api-access-78cpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:58:13.721035 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.721015    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78cpw\" (UniqueName: \"kubernetes.io/projected/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-kube-api-access-78cpw\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:58:13.721035 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.721034    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-tls-certs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:58:13.721202 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.721045    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-home\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:58:13.721202 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.721053    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-model-cache\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:58:13.721202 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.721060    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-dshm\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:58:13.721202 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:13.721078    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5-tmp-dir\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 19:58:14.148349 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:14.148314    2572 generic.go:356] "Generic (PLEG): container finished" podID="ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5" containerID="1fcf11d5f36e225c48bb9c7699106077298377f934f721b03692b64e09c98dcf" exitCode=0
Jul 28 19:58:14.148509 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:14.148365    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"
Jul 28 19:58:14.148509 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:14.148379    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr" event={"ID":"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5","Type":"ContainerDied","Data":"1fcf11d5f36e225c48bb9c7699106077298377f934f721b03692b64e09c98dcf"}
Jul 28 19:58:14.148509 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:14.148411    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr" event={"ID":"ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5","Type":"ContainerDied","Data":"aa6b8ce07a0b8af0480705233024f7d0270557f02c44de4be4f982fd50595cda"}
Jul 28 19:58:14.148509 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:14.148426    2572 scope.go:117] "RemoveContainer" containerID="1fcf11d5f36e225c48bb9c7699106077298377f934f721b03692b64e09c98dcf"
Jul 28 19:58:14.157254 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:14.157235    2572 scope.go:117] "RemoveContainer" containerID="1fcf11d5f36e225c48bb9c7699106077298377f934f721b03692b64e09c98dcf"
Jul 28 19:58:14.157491 ip-10-0-138-37 kubenswrapper[2572]: E0728 19:58:14.157465    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fcf11d5f36e225c48bb9c7699106077298377f934f721b03692b64e09c98dcf\": container with ID starting with 1fcf11d5f36e225c48bb9c7699106077298377f934f721b03692b64e09c98dcf not found: ID does not exist" containerID="1fcf11d5f36e225c48bb9c7699106077298377f934f721b03692b64e09c98dcf"
Jul 28 19:58:14.157538 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:14.157500    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fcf11d5f36e225c48bb9c7699106077298377f934f721b03692b64e09c98dcf"} err="failed to get container status \"1fcf11d5f36e225c48bb9c7699106077298377f934f721b03692b64e09c98dcf\": rpc error: code = NotFound desc = could not find container \"1fcf11d5f36e225c48bb9c7699106077298377f934f721b03692b64e09c98dcf\": container with ID starting with 1fcf11d5f36e225c48bb9c7699106077298377f934f721b03692b64e09c98dcf not found: ID does not exist"
Jul 28 19:58:14.180953 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:14.180929    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"]
Jul 28 19:58:14.183994 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:14.183976    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-6e7e0dcd-kserve-6c8c85f4c-nktkr"]
Jul 28 19:58:14.864018 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:58:14.863990    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5" path="/var/lib/kubelet/pods/ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5/volumes"
Jul 28 19:59:34.883309 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:59:34.883276    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 19:59:34.887311 ip-10-0-138-37 kubenswrapper[2572]: I0728 19:59:34.887286    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 20:01:14.001255 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:01:14.001225    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:25.922091 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:01:25.922038    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:25.981199 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:01:25.981164    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:37.910513 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:01:37.910482    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:42.398220 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:01:42.398186    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:42.533288 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:01:42.533255    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:43.035231 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:01:43.035202    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:48.690558 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:01:48.690511    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:48.751474 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:01:48.751430    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:48.906844 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:01:48.906815    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:49.324052 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:01:49.324024    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:49.425252 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:01:49.425218    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:54.484961 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:01:54.484926    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:54.583753 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:01:54.583720    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:54.682617 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:01:54.682592    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:55.163792 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:01:55.163746    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:55.262788 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:01:55.262757    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:19.974549 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:19.974505    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8"]
Jul 28 20:02:19.975112 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:19.974950    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5" containerName="main"
Jul 28 20:02:19.975112 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:19.974963    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5" containerName="main"
Jul 28 20:02:19.975112 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:19.975031    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffbf99bc-98ef-46b5-9be9-7c9ec4b90ec5" containerName="main"
Jul 28 20:02:19.978315 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:19.978294    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8"
Jul 28 20:02:19.981103 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:19.981052    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6zs4\""
Jul 28 20:02:19.985023 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:19.984993    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8"]
Jul 28 20:02:20.084278 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:20.084254    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8\" (UID: \"9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8"
Jul 28 20:02:20.185554 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:20.185524    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8\" (UID: \"9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8"
Jul 28 20:02:20.186184 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:20.186161    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8\" (UID: \"9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8"
Jul 28 20:02:20.293901 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:20.293839    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8"
Jul 28 20:02:20.422392 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:20.422362    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8"]
Jul 28 20:02:20.424268 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:02:20.424243    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ce2ac3f_8557_4b33_a908_7fbb39a1f3e8.slice/crio-3efbfb2574b553c4fad1d6b6242332eebb0a490b1afed67ade75c22428793eea WatchSource:0}: Error finding container 3efbfb2574b553c4fad1d6b6242332eebb0a490b1afed67ade75c22428793eea: Status 404 returned error can't find the container with id 3efbfb2574b553c4fad1d6b6242332eebb0a490b1afed67ade75c22428793eea
Jul 28 20:02:20.426575 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:20.426557    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:02:20.486360 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:20.486331    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8" event={"ID":"9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8","Type":"ContainerStarted","Data":"3efbfb2574b553c4fad1d6b6242332eebb0a490b1afed67ade75c22428793eea"}
Jul 28 20:02:24.509935 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:24.509869    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8" event={"ID":"9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8","Type":"ContainerStarted","Data":"f27375ae1b7d72d77e89ccd69b6e2ea8a266c28140063aa2454b1ec69ab4b920"}
Jul 28 20:02:24.527606 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:24.527560    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8" podStartSLOduration=1.882639599 podStartE2EDuration="5.527544246s" totalImagesPullingTime="3.644904647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:02:24.525706475 +0000 UTC m=+1970.113369441" watchObservedRunningTime="2026-07-28 20:02:24.527544246 +0000 UTC m=+1970.115207191"
Jul 28 20:02:25.516148 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:25.516119    2572 generic.go:356] "Generic (PLEG): container finished" podID="9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8" containerID="f27375ae1b7d72d77e89ccd69b6e2ea8a266c28140063aa2454b1ec69ab4b920" exitCode=0
Jul 28 20:02:25.516518 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:25.516207    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8" event={"ID":"9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8","Type":"ContainerDied","Data":"f27375ae1b7d72d77e89ccd69b6e2ea8a266c28140063aa2454b1ec69ab4b920"}
Jul 28 20:02:26.655914 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:26.655893    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8"
Jul 28 20:02:26.738298 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:26.738245    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8-maas-api-service-ca\") pod \"9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8\" (UID: \"9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8\") "
Jul 28 20:02:26.738583 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:26.738562    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8" (UID: "9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:02:26.839237 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:26.839209    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8-maas-api-service-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:02:27.528932 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:27.528897    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8" event={"ID":"9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8","Type":"ContainerDied","Data":"3efbfb2574b553c4fad1d6b6242332eebb0a490b1afed67ade75c22428793eea"}
Jul 28 20:02:27.528932 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:27.528919    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8"
Jul 28 20:02:27.529137 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:27.528974    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3efbfb2574b553c4fad1d6b6242332eebb0a490b1afed67ade75c22428793eea"
Jul 28 20:02:35.159062 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:35.159034    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:35.346435 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:35.346401    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8"]
Jul 28 20:02:35.349062 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:35.349039    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8a457bc8-jmlv8"]
Jul 28 20:02:36.865883 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:36.865847    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8" path="/var/lib/kubelet/pods/9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8/volumes"
Jul 28 20:02:36.991946 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:36.991912    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:38.756209 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:38.756174    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:38.941369 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:38.941340    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:39.145715 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:39.145681    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:39.414037 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:39.413970    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:39.526221 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:02:39.526200    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:10.267973 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:10.267942    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:10.368169 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:10.368134    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:10.442469 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:10.442436    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:10.522760 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:10.522695    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:10.591726 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:10.591699    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:10.930874 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:10.930846    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:11.056385 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:11.056364    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:11.361800 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:11.361767    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:11.839346 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:11.839319    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:13.797638 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:13.797590    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:13.937388 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:13.937358    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:14.036535 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:14.036508    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:14.137327 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:14.137283    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:14.380953 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:14.380924    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:14.487938 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:14.487831    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:38.937827 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:38.937791    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:44.635916 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:44.635874    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:44.700803 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:44.700760    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:44.765327 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:44.765290    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:44.835376 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:44.835340    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:45.957338 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:45.957301    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:46.177941 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:46.177903    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:46.397594 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:46.397559    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:46.723035 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:46.722945    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:47.630229 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:47.630196    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:49.057957 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:49.057929    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:49.137093 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:49.135682    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:49.200587 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:49.200554    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:49.560387 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:49.560361    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:49.807519 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:49.807491    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:49.906989 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:03:49.906904    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:19.846512 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:04:19.846470    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:19.944342 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:04:19.944310    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:20.007963 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:04:20.007907    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:20.114994 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:04:20.114902    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:20.333990 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:04:20.333946    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:20.793951 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:04:20.793917    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:20.899259 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:04:20.899227    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:20.992447 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:04:20.992412    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:22.404457 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:04:22.404411    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:28.424791 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:04:28.424758    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:28.529178 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:04:28.529144    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:28.624909 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:04:28.624880    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:28.725088 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:04:28.724998    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:29.153922 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:04:29.153888    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:34.921549 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:04:34.921519    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 20:04:34.927862 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:04:34.927841    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 20:04:55.185963 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:04:55.185922    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:02.016181 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:02.016138    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:02.167914 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:02.167877    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:02.293549 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:02.293484    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:10.612881 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:10.612835    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:10.714677 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:10.714650    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:17.311607 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:17.311563    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:18.149663 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:18.149623    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:24.139412 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:24.139377    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:24.385973 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:24.385944    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:34.767684 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:34.767597    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:35.868613 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:35.868578    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:41.914158 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:41.914120    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:42.020515 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:42.020484    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:42.115365 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:42.115338    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:42.213649 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:42.213580    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:49.630826 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:49.630792    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:49.728473 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:49.728445    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:50.697673 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:50.697645    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:51.556709 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:51.556674    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:57.362870 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:57.362840    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:57.436919 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:57.436886    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:58.063468 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:05:58.063424    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:03.650739 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:03.650699    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:03.751009 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:03.750982    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:03.922056 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:03.921995    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:04.220167 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:04.220089    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:04.829557 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:04.829521    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:07.222853 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:07.222820    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:07.309760 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:07.309730    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:07.379420 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:07.379395    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:07.630236 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:07.630205    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:07.952715 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:07.952650    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:08.068603 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:08.068573    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:32.239326 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:32.239288    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:32.897217 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:32.897183    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:32.983063 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:32.983032    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:33.328657 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:33.328624    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:39.519795 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:39.519758    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:39.622180 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:39.622149    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:50.265603 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:50.265572    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:50.360242 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:50.360211    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:50.521744 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:50.521669    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:50.581366 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:50.581332    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:50.944730 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:50.944699    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:56.599770 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:56.599727    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:56.655613 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:56.655578    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:56.854924 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:56.854839    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:57.068842 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:57.068807    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:57.202303 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:57.202213    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:57.297731 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:57.297700    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:57.414842 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:06:57.414819    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:03.235180 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:07:03.235141    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:03.339673 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:07:03.339631    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:03.434767 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:07:03.434741    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:03.581458 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:07:03.581432    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:09.521485 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:07:09.521431    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:09.601159 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:07:09.601129    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:33.359631 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:07:33.359598    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:33.652807 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:07:33.652711    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:33.953542 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:07:33.953426    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:36.613953 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:07:36.613896    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:36.715402 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:07:36.715369    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:36.815243 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:07:36.815219    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:37.288340 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:07:37.288311    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:38.972098 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:07:38.972047    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:08:01.565553 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:01.565512    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:08:09.885133 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:09.885091    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"]
Jul 28 20:08:09.885686 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:09.885657    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8" containerName="revoke-keys"
Jul 28 20:08:09.885686 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:09.885686    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8" containerName="revoke-keys"
Jul 28 20:08:09.885845 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:09.885793    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ce2ac3f-8557-4b33-a908-7fbb39a1f3e8" containerName="revoke-keys"
Jul 28 20:08:09.888666 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:09.888650    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:09.892244 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:09.892221    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-89688d3f\"/\"default-dockercfg-gqxf6\""
Jul 28 20:08:09.892381 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:09.892281    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-89688d3f\"/\"openshift-service-ca.crt\""
Jul 28 20:08:09.892381 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:09.892281    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-89688d3f\"/\"e2e-lifecycle-model-89688d3f-kserve-self-signed-certs\""
Jul 28 20:08:09.892510 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:09.892490    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-89688d3f\"/\"kube-root-ca.crt\""
Jul 28 20:08:09.899228 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:09.899204    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"]
Jul 28 20:08:09.919400 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:09.919378    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:08:09.933528 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:09.933503    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-tmp-dir\") pod \"e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") " pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:09.933621 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:09.933539    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-dshm\") pod \"e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") " pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:09.933621 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:09.933562    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kf5m\" (UniqueName: \"kubernetes.io/projected/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-kube-api-access-8kf5m\") pod \"e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") " pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:09.933738 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:09.933680    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-home\") pod \"e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") " pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:09.933794 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:09.933749    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-model-cache\") pod \"e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") " pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:09.933846 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:09.933808    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-tls-certs\") pod \"e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") " pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:10.034747 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:10.034719    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-dshm\") pod \"e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") " pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:10.034886 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:10.034750    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kf5m\" (UniqueName: \"kubernetes.io/projected/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-kube-api-access-8kf5m\") pod \"e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") " pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:10.034886 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:10.034783    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-home\") pod \"e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") " pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:10.034886 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:10.034837    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-model-cache\") pod \"e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") " pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:10.035053 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:10.034891    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-tls-certs\") pod \"e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") " pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:10.035053 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:10.034920    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-tmp-dir\") pod \"e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") " pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:10.035300 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:10.035278    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-home\") pod \"e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") " pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:10.035300 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:10.035291    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-tmp-dir\") pod \"e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") " pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:10.035434 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:10.035379    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-model-cache\") pod \"e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") " pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:10.036962 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:10.036935    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-dshm\") pod \"e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") " pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:10.037221 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:10.037205    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-tls-certs\") pod \"e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") " pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:10.042757 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:10.042736    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kf5m\" (UniqueName: \"kubernetes.io/projected/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-kube-api-access-8kf5m\") pod \"e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") " pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:10.201171 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:10.201115    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:10.330316 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:10.330288    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"]
Jul 28 20:08:10.331760 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:08:10.331733    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1729ce76_ae8a_423c_81e0_7b6cc3e85e94.slice/crio-61ee972cc8c9418464cf7a982f9e35f0d20e5d93dbb47ada1811fbf63495137f WatchSource:0}: Error finding container 61ee972cc8c9418464cf7a982f9e35f0d20e5d93dbb47ada1811fbf63495137f: Status 404 returned error can't find the container with id 61ee972cc8c9418464cf7a982f9e35f0d20e5d93dbb47ada1811fbf63495137f
Jul 28 20:08:10.407897 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:10.407870    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z" event={"ID":"1729ce76-ae8a-423c-81e0-7b6cc3e85e94","Type":"ContainerStarted","Data":"756d8582fe5492a065b6b989c2c9fbd13aafc988e11c861f10f408a2811cb4be"}
Jul 28 20:08:10.408002 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:10.407912    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z" event={"ID":"1729ce76-ae8a-423c-81e0-7b6cc3e85e94","Type":"ContainerStarted","Data":"61ee972cc8c9418464cf7a982f9e35f0d20e5d93dbb47ada1811fbf63495137f"}
Jul 28 20:08:10.431448 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:10.431405    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z" podStartSLOduration=1.431389854 podStartE2EDuration="1.431389854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:08:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:08:10.42863156 +0000 UTC m=+2316.016294505" watchObservedRunningTime="2026-07-28 20:08:10.431389854 +0000 UTC m=+2316.019052798"
Jul 28 20:08:20.201864 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:20.201832    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:20.202282 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:20.201882    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:20.213764 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:20.213740    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:20.471460 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:20.471397    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:08:35.009246 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:35.009218    2572 scope.go:117] "RemoveContainer" containerID="f27375ae1b7d72d77e89ccd69b6e2ea8a266c28140063aa2454b1ec69ab4b920"
Jul 28 20:08:56.446678 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:56.446643    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:08:56.511672 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:56.511644    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:08:56.574536 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:56.574509    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:08:56.635767 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:56.635738    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:08:59.789127 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:08:59.789090    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:09:04.774172 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:04.774123    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"]
Jul 28 20:09:04.868531 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:04.868501    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:09:19.792793 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:19.792749    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z" podUID="1729ce76-ae8a-423c-81e0-7b6cc3e85e94" containerName="main" containerID="cri-o://756d8582fe5492a065b6b989c2c9fbd13aafc988e11c861f10f408a2811cb4be" gracePeriod=45
Jul 28 20:09:20.038278 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.038257    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:09:20.086853 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.086008    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-dshm\") pod \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") "
Jul 28 20:09:20.086853 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.086097    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-model-cache\") pod \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") "
Jul 28 20:09:20.086853 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.086145    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-tls-certs\") pod \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") "
Jul 28 20:09:20.086853 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.086173    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-home\") pod \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") "
Jul 28 20:09:20.086853 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.086233    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8kf5m\" (UniqueName: \"kubernetes.io/projected/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-kube-api-access-8kf5m\") pod \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") "
Jul 28 20:09:20.086853 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.086272    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-tmp-dir\") pod \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\" (UID: \"1729ce76-ae8a-423c-81e0-7b6cc3e85e94\") "
Jul 28 20:09:20.087650 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.087345    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1729ce76-ae8a-423c-81e0-7b6cc3e85e94" (UID: "1729ce76-ae8a-423c-81e0-7b6cc3e85e94"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:09:20.094494 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.094393    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-model-cache" (OuterVolumeSpecName: "model-cache") pod "1729ce76-ae8a-423c-81e0-7b6cc3e85e94" (UID: "1729ce76-ae8a-423c-81e0-7b6cc3e85e94"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:09:20.094494 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.094439    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-home" (OuterVolumeSpecName: "home") pod "1729ce76-ae8a-423c-81e0-7b6cc3e85e94" (UID: "1729ce76-ae8a-423c-81e0-7b6cc3e85e94"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:09:20.096420 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.096394    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-dshm" (OuterVolumeSpecName: "dshm") pod "1729ce76-ae8a-423c-81e0-7b6cc3e85e94" (UID: "1729ce76-ae8a-423c-81e0-7b6cc3e85e94"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:09:20.096607 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.096580    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1729ce76-ae8a-423c-81e0-7b6cc3e85e94" (UID: "1729ce76-ae8a-423c-81e0-7b6cc3e85e94"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:09:20.096831 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.096800    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-kube-api-access-8kf5m" (OuterVolumeSpecName: "kube-api-access-8kf5m") pod "1729ce76-ae8a-423c-81e0-7b6cc3e85e94" (UID: "1729ce76-ae8a-423c-81e0-7b6cc3e85e94"). InnerVolumeSpecName "kube-api-access-8kf5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:09:20.187041 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.187014    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-dshm\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:09:20.187041 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.187038    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-model-cache\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:09:20.187221 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.187049    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-tls-certs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:09:20.187221 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.187057    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-home\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:09:20.187221 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.187080    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kf5m\" (UniqueName: \"kubernetes.io/projected/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-kube-api-access-8kf5m\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:09:20.187221 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.187091    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1729ce76-ae8a-423c-81e0-7b6cc3e85e94-tmp-dir\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:09:20.768947 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.768909    2572 generic.go:356] "Generic (PLEG): container finished" podID="1729ce76-ae8a-423c-81e0-7b6cc3e85e94" containerID="756d8582fe5492a065b6b989c2c9fbd13aafc988e11c861f10f408a2811cb4be" exitCode=0
Jul 28 20:09:20.769150 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.768964    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"
Jul 28 20:09:20.769150 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.768998    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z" event={"ID":"1729ce76-ae8a-423c-81e0-7b6cc3e85e94","Type":"ContainerDied","Data":"756d8582fe5492a065b6b989c2c9fbd13aafc988e11c861f10f408a2811cb4be"}
Jul 28 20:09:20.769150 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.769044    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z" event={"ID":"1729ce76-ae8a-423c-81e0-7b6cc3e85e94","Type":"ContainerDied","Data":"61ee972cc8c9418464cf7a982f9e35f0d20e5d93dbb47ada1811fbf63495137f"}
Jul 28 20:09:20.769150 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.769079    2572 scope.go:117] "RemoveContainer" containerID="756d8582fe5492a065b6b989c2c9fbd13aafc988e11c861f10f408a2811cb4be"
Jul 28 20:09:20.778827 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.778811    2572 scope.go:117] "RemoveContainer" containerID="756d8582fe5492a065b6b989c2c9fbd13aafc988e11c861f10f408a2811cb4be"
Jul 28 20:09:20.779128 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:09:20.779109    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"756d8582fe5492a065b6b989c2c9fbd13aafc988e11c861f10f408a2811cb4be\": container with ID starting with 756d8582fe5492a065b6b989c2c9fbd13aafc988e11c861f10f408a2811cb4be not found: ID does not exist" containerID="756d8582fe5492a065b6b989c2c9fbd13aafc988e11c861f10f408a2811cb4be"
Jul 28 20:09:20.779182 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.779135    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"756d8582fe5492a065b6b989c2c9fbd13aafc988e11c861f10f408a2811cb4be"} err="failed to get container status \"756d8582fe5492a065b6b989c2c9fbd13aafc988e11c861f10f408a2811cb4be\": rpc error: code = NotFound desc = could not find container \"756d8582fe5492a065b6b989c2c9fbd13aafc988e11c861f10f408a2811cb4be\": container with ID starting with 756d8582fe5492a065b6b989c2c9fbd13aafc988e11c861f10f408a2811cb4be not found: ID does not exist"
Jul 28 20:09:20.790518 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.790482    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"]
Jul 28 20:09:20.796253 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.796231    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-89688d3f/e2e-lifecycle-model-89688d3f-kserve-7749fbfbcf-tqq6z"]
Jul 28 20:09:20.864934 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:20.864897    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1729ce76-ae8a-423c-81e0-7b6cc3e85e94" path="/var/lib/kubelet/pods/1729ce76-ae8a-423c-81e0-7b6cc3e85e94/volumes"
Jul 28 20:09:34.956548 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:34.956517    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 20:09:34.963550 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:34.963526    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 20:09:41.086708 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:41.086648    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:09:41.945387 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:41.945341    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:09:47.578308 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:09:47.578271    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:10.295958 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:10.295920    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:10.361275 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:10.361245    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:10.578498 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:10.578467    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:10.799917 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:10.799887    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:11.926645 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:11.926606    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:12.026171 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:12.026141    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:12.124166 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:12.124124    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:12.916441 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:12.916411    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:12.998137 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:12.998103    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:13.100673 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:13.100644    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:14.470947 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:14.470875    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:20.303011 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:20.302974    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:20.370533 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:20.370506    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:21.013325 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:21.013302    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:27.182578 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:27.182539    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:27.290892 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:27.290863    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:27.383147 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:27.383121    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:37.869536 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:37.869500    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:38.202090 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:38.201989    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:59.150364 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:10:59.150321    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:01.392139 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:01.392103    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:01.493392 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:01.493365    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:05.645887 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:05.645853    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:05.746195 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:05.746170    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:06.011185 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:06.011101    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:06.323329 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:06.323308    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:06.499520 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:06.499482    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:09.570974 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:09.570927    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:09.649724 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:09.649692    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:09.719123 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:09.719100    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:09.865020 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:09.864936    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:10.171504 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:10.171432    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:10.273625 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:10.273596    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:10.597487 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:10.597455    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:27.604604 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:27.604500    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:27.656534 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:27.656495    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:34.574704 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:34.574662    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:34.914347 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:34.914262    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:35.226505 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:35.226431    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:35.528125 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:35.528020    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:37.750559 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:37.750524    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:37.838655 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:37.838624    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:37.902465 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:37.902444    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:38.339999 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:38.339970    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:38.430035 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:38.430006    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:56.972666 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:11:56.972622    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:12:02.707487 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:02.707455    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:12:16.904192 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:16.904154    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"]
Jul 28 20:12:16.904628 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:16.904522    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1729ce76-ae8a-423c-81e0-7b6cc3e85e94" containerName="main"
Jul 28 20:12:16.904628 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:16.904533    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1729ce76-ae8a-423c-81e0-7b6cc3e85e94" containerName="main"
Jul 28 20:12:16.904628 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:16.904590    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1729ce76-ae8a-423c-81e0-7b6cc3e85e94" containerName="main"
Jul 28 20:12:16.907500 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:16.907482    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:16.911199 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:16.911154    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-68aa77\"/\"default-dockercfg-7h65b\""
Jul 28 20:12:16.911199 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:16.911180    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-68aa77\"/\"test-model-68aa77-kserve-self-signed-certs\""
Jul 28 20:12:16.911429 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:16.911184    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-68aa77\"/\"kube-root-ca.crt\""
Jul 28 20:12:16.911429 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:16.911206    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-68aa77\"/\"openshift-service-ca.crt\""
Jul 28 20:12:16.917503 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:16.917482    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"]
Jul 28 20:12:16.979652 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:16.979621    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njl4q\" (UniqueName: \"kubernetes.io/projected/50760b3f-80de-423f-bb38-2f8f0f404a0b-kube-api-access-njl4q\") pod \"test-model-68aa77-kserve-ccc86984f-gjt85\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") " pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:16.979790 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:16.979663    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-home\") pod \"test-model-68aa77-kserve-ccc86984f-gjt85\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") " pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:16.979790 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:16.979687    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50760b3f-80de-423f-bb38-2f8f0f404a0b-tls-certs\") pod \"test-model-68aa77-kserve-ccc86984f-gjt85\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") " pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:16.979790 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:16.979760    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-tmp-dir\") pod \"test-model-68aa77-kserve-ccc86984f-gjt85\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") " pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:16.979790 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:16.979783    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-dshm\") pod \"test-model-68aa77-kserve-ccc86984f-gjt85\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") " pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:16.979960 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:16.979813    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-model-cache\") pod \"test-model-68aa77-kserve-ccc86984f-gjt85\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") " pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:16.994999 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:16.994972    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:12:17.080373 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:17.080348    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-tmp-dir\") pod \"test-model-68aa77-kserve-ccc86984f-gjt85\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") " pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:17.080487 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:17.080376    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-dshm\") pod \"test-model-68aa77-kserve-ccc86984f-gjt85\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") " pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:17.080487 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:17.080412    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-model-cache\") pod \"test-model-68aa77-kserve-ccc86984f-gjt85\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") " pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:17.080487 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:17.080453    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njl4q\" (UniqueName: \"kubernetes.io/projected/50760b3f-80de-423f-bb38-2f8f0f404a0b-kube-api-access-njl4q\") pod \"test-model-68aa77-kserve-ccc86984f-gjt85\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") " pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:17.080487 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:17.080475    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-home\") pod \"test-model-68aa77-kserve-ccc86984f-gjt85\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") " pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:17.080487 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:17.080489    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50760b3f-80de-423f-bb38-2f8f0f404a0b-tls-certs\") pod \"test-model-68aa77-kserve-ccc86984f-gjt85\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") " pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:17.080800 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:17.080778    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-tmp-dir\") pod \"test-model-68aa77-kserve-ccc86984f-gjt85\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") " pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:17.080878 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:17.080856    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-model-cache\") pod \"test-model-68aa77-kserve-ccc86984f-gjt85\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") " pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:17.080915 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:17.080899    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-home\") pod \"test-model-68aa77-kserve-ccc86984f-gjt85\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") " pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:17.082589 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:17.082565    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-dshm\") pod \"test-model-68aa77-kserve-ccc86984f-gjt85\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") " pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:17.082807 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:17.082790    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50760b3f-80de-423f-bb38-2f8f0f404a0b-tls-certs\") pod \"test-model-68aa77-kserve-ccc86984f-gjt85\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") " pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:17.088533 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:17.088510    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njl4q\" (UniqueName: \"kubernetes.io/projected/50760b3f-80de-423f-bb38-2f8f0f404a0b-kube-api-access-njl4q\") pod \"test-model-68aa77-kserve-ccc86984f-gjt85\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") " pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:17.218014 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:17.217951    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:17.356957 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:17.356931    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"]
Jul 28 20:12:17.358117 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:12:17.358087    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50760b3f_80de_423f_bb38_2f8f0f404a0b.slice/crio-f161f172861a27d3be5d3ff56eb64d61cea9824cde11b4ad95c52174e7910bf4 WatchSource:0}: Error finding container f161f172861a27d3be5d3ff56eb64d61cea9824cde11b4ad95c52174e7910bf4: Status 404 returned error can't find the container with id f161f172861a27d3be5d3ff56eb64d61cea9824cde11b4ad95c52174e7910bf4
Jul 28 20:12:17.730288 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:17.730241    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85" event={"ID":"50760b3f-80de-423f-bb38-2f8f0f404a0b","Type":"ContainerStarted","Data":"8b22a2c3a6cbd67e128a89c063e08ceee0b2a87a3301bb5eabf346767589a4f6"}
Jul 28 20:12:17.730473 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:17.730308    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85" event={"ID":"50760b3f-80de-423f-bb38-2f8f0f404a0b","Type":"ContainerStarted","Data":"f161f172861a27d3be5d3ff56eb64d61cea9824cde11b4ad95c52174e7910bf4"}
Jul 28 20:12:17.749663 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:17.749611    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85" podStartSLOduration=1.749596029 podStartE2EDuration="1.749596029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:12:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:12:17.747063216 +0000 UTC m=+2563.334726159" watchObservedRunningTime="2026-07-28 20:12:17.749596029 +0000 UTC m=+2563.337258973"
Jul 28 20:12:27.218295 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:27.218257    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:27.218957 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:27.218313    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:27.230618 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:27.230591    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:27.800440 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:27.800413    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:12:47.797004 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:47.796972    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"]
Jul 28 20:12:47.800255 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:47.800232    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:47.802864 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:47.802843    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e98043\"/\"default-dockercfg-hlnxd\""
Jul 28 20:12:47.802864 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:47.802852    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e98043\"/\"openshift-service-ca.crt\""
Jul 28 20:12:47.804009 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:47.803989    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e98043\"/\"kube-root-ca.crt\""
Jul 28 20:12:47.806500 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:47.806482    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e98043\"/\"test-model-e98043-kserve-self-signed-certs\""
Jul 28 20:12:47.810529 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:47.810504    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"]
Jul 28 20:12:47.896993 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:47.896970    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:12:47.926466 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:47.926444    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-home\") pod \"test-model-e98043-kserve-7666d988dc-mxmpw\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") " pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:47.926562 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:47.926471    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78zxr\" (UniqueName: \"kubernetes.io/projected/a0a48b16-f022-4df8-a231-80f281e5ffcc-kube-api-access-78zxr\") pod \"test-model-e98043-kserve-7666d988dc-mxmpw\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") " pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:47.926562 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:47.926494    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0a48b16-f022-4df8-a231-80f281e5ffcc-tls-certs\") pod \"test-model-e98043-kserve-7666d988dc-mxmpw\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") " pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:47.926562 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:47.926512    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-dshm\") pod \"test-model-e98043-kserve-7666d988dc-mxmpw\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") " pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:47.926665 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:47.926593    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-model-cache\") pod \"test-model-e98043-kserve-7666d988dc-mxmpw\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") " pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:47.926665 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:47.926621    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-tmp-dir\") pod \"test-model-e98043-kserve-7666d988dc-mxmpw\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") " pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:48.027762 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:48.027737    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-home\") pod \"test-model-e98043-kserve-7666d988dc-mxmpw\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") " pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:48.027881 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:48.027772    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78zxr\" (UniqueName: \"kubernetes.io/projected/a0a48b16-f022-4df8-a231-80f281e5ffcc-kube-api-access-78zxr\") pod \"test-model-e98043-kserve-7666d988dc-mxmpw\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") " pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:48.027881 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:48.027794    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0a48b16-f022-4df8-a231-80f281e5ffcc-tls-certs\") pod \"test-model-e98043-kserve-7666d988dc-mxmpw\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") " pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:48.027881 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:48.027818    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-dshm\") pod \"test-model-e98043-kserve-7666d988dc-mxmpw\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") " pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:48.028044 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:48.027901    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-model-cache\") pod \"test-model-e98043-kserve-7666d988dc-mxmpw\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") " pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:48.028044 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:48.027947    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-tmp-dir\") pod \"test-model-e98043-kserve-7666d988dc-mxmpw\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") " pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:48.028189 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:48.028163    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-home\") pod \"test-model-e98043-kserve-7666d988dc-mxmpw\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") " pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:48.028343 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:48.028311    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-tmp-dir\") pod \"test-model-e98043-kserve-7666d988dc-mxmpw\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") " pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:48.028420 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:48.028383    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-model-cache\") pod \"test-model-e98043-kserve-7666d988dc-mxmpw\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") " pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:48.030331 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:48.030307    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0a48b16-f022-4df8-a231-80f281e5ffcc-tls-certs\") pod \"test-model-e98043-kserve-7666d988dc-mxmpw\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") " pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:48.030415 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:48.030316    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-dshm\") pod \"test-model-e98043-kserve-7666d988dc-mxmpw\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") " pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:48.036431 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:48.036410    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78zxr\" (UniqueName: \"kubernetes.io/projected/a0a48b16-f022-4df8-a231-80f281e5ffcc-kube-api-access-78zxr\") pod \"test-model-e98043-kserve-7666d988dc-mxmpw\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") " pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:48.111496 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:48.111469    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:48.243010 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:48.242982    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"]
Jul 28 20:12:48.244433 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:12:48.244407    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0a48b16_f022_4df8_a231_80f281e5ffcc.slice/crio-d8c9501247faaf5ec0998beaca7a84084d1dfc7b0dfb4fadbc16554da6c52292 WatchSource:0}: Error finding container d8c9501247faaf5ec0998beaca7a84084d1dfc7b0dfb4fadbc16554da6c52292: Status 404 returned error can't find the container with id d8c9501247faaf5ec0998beaca7a84084d1dfc7b0dfb4fadbc16554da6c52292
Jul 28 20:12:48.907332 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:48.907300    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw" event={"ID":"a0a48b16-f022-4df8-a231-80f281e5ffcc","Type":"ContainerStarted","Data":"6669a5e540f63f4657938ed93ad4732f5bcfc695502bebc166f04d9bc8274d70"}
Jul 28 20:12:48.907710 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:48.907340    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw" event={"ID":"a0a48b16-f022-4df8-a231-80f281e5ffcc","Type":"ContainerStarted","Data":"d8c9501247faaf5ec0998beaca7a84084d1dfc7b0dfb4fadbc16554da6c52292"}
Jul 28 20:12:48.927259 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:48.927218    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw" podStartSLOduration=1.9272051879999998 podStartE2EDuration="1.927205188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:12:48.9243814 +0000 UTC m=+2594.512044344" watchObservedRunningTime="2026-07-28 20:12:48.927205188 +0000 UTC m=+2594.514868133"
Jul 28 20:12:58.112015 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:58.111935    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:58.112015 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:58.111979    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:58.123779 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:58.123755    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:12:58.969240 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:12:58.969212    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:13:32.680503 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:32.680462    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"]
Jul 28 20:13:32.701656 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:32.701630    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:33.705559 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:33.705506    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"]
Jul 28 20:13:33.720163 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:33.720132    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:43.435416 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:43.435380    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:44.325674 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:44.325640    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:44.395554 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:44.395512    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:44.459130 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:44.459103    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:44.559543 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:44.559509    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:44.944449 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:44.944417    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:47.699357 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:47.699319    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85" podUID="50760b3f-80de-423f-bb38-2f8f0f404a0b" containerName="main" containerID="cri-o://8b22a2c3a6cbd67e128a89c063e08ceee0b2a87a3301bb5eabf346767589a4f6" gracePeriod=45
Jul 28 20:13:47.790229 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:47.790196    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85" podUID="50760b3f-80de-423f-bb38-2f8f0f404a0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": dial tcp 10.132.0.55:8000: connect: connection refused"
Jul 28 20:13:47.934864 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:47.934832    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:13:48.014254 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.014195    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njl4q\" (UniqueName: \"kubernetes.io/projected/50760b3f-80de-423f-bb38-2f8f0f404a0b-kube-api-access-njl4q\") pod \"50760b3f-80de-423f-bb38-2f8f0f404a0b\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") "
Jul 28 20:13:48.014254 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.014239    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-dshm\") pod \"50760b3f-80de-423f-bb38-2f8f0f404a0b\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") "
Jul 28 20:13:48.014419 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.014264    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50760b3f-80de-423f-bb38-2f8f0f404a0b-tls-certs\") pod \"50760b3f-80de-423f-bb38-2f8f0f404a0b\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") "
Jul 28 20:13:48.014419 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.014282    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-model-cache\") pod \"50760b3f-80de-423f-bb38-2f8f0f404a0b\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") "
Jul 28 20:13:48.014419 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.014319    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-home\") pod \"50760b3f-80de-423f-bb38-2f8f0f404a0b\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") "
Jul 28 20:13:48.014419 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.014342    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-tmp-dir\") pod \"50760b3f-80de-423f-bb38-2f8f0f404a0b\" (UID: \"50760b3f-80de-423f-bb38-2f8f0f404a0b\") "
Jul 28 20:13:48.014621 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.014593    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-model-cache" (OuterVolumeSpecName: "model-cache") pod "50760b3f-80de-423f-bb38-2f8f0f404a0b" (UID: "50760b3f-80de-423f-bb38-2f8f0f404a0b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:13:48.014678 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.014631    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-home" (OuterVolumeSpecName: "home") pod "50760b3f-80de-423f-bb38-2f8f0f404a0b" (UID: "50760b3f-80de-423f-bb38-2f8f0f404a0b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:13:48.014678 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.014647    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "50760b3f-80de-423f-bb38-2f8f0f404a0b" (UID: "50760b3f-80de-423f-bb38-2f8f0f404a0b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:13:48.016514 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.016490    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50760b3f-80de-423f-bb38-2f8f0f404a0b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "50760b3f-80de-423f-bb38-2f8f0f404a0b" (UID: "50760b3f-80de-423f-bb38-2f8f0f404a0b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:13:48.016615 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.016489    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-dshm" (OuterVolumeSpecName: "dshm") pod "50760b3f-80de-423f-bb38-2f8f0f404a0b" (UID: "50760b3f-80de-423f-bb38-2f8f0f404a0b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:13:48.016615 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.016527    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50760b3f-80de-423f-bb38-2f8f0f404a0b-kube-api-access-njl4q" (OuterVolumeSpecName: "kube-api-access-njl4q") pod "50760b3f-80de-423f-bb38-2f8f0f404a0b" (UID: "50760b3f-80de-423f-bb38-2f8f0f404a0b"). InnerVolumeSpecName "kube-api-access-njl4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:13:48.115678 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.115651    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-home\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:13:48.115782 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.115699    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-tmp-dir\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:13:48.115782 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.115714    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njl4q\" (UniqueName: \"kubernetes.io/projected/50760b3f-80de-423f-bb38-2f8f0f404a0b-kube-api-access-njl4q\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:13:48.115782 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.115727    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-dshm\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:13:48.115782 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.115740    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50760b3f-80de-423f-bb38-2f8f0f404a0b-tls-certs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:13:48.115782 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.115752    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50760b3f-80de-423f-bb38-2f8f0f404a0b-model-cache\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:13:48.231981 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.231952    2572 generic.go:356] "Generic (PLEG): container finished" podID="50760b3f-80de-423f-bb38-2f8f0f404a0b" containerID="8b22a2c3a6cbd67e128a89c063e08ceee0b2a87a3301bb5eabf346767589a4f6" exitCode=0
Jul 28 20:13:48.232141 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.231999    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"
Jul 28 20:13:48.232141 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.232019    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85" event={"ID":"50760b3f-80de-423f-bb38-2f8f0f404a0b","Type":"ContainerDied","Data":"8b22a2c3a6cbd67e128a89c063e08ceee0b2a87a3301bb5eabf346767589a4f6"}
Jul 28 20:13:48.232141 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.232057    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85" event={"ID":"50760b3f-80de-423f-bb38-2f8f0f404a0b","Type":"ContainerDied","Data":"f161f172861a27d3be5d3ff56eb64d61cea9824cde11b4ad95c52174e7910bf4"}
Jul 28 20:13:48.232141 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.232099    2572 scope.go:117] "RemoveContainer" containerID="8b22a2c3a6cbd67e128a89c063e08ceee0b2a87a3301bb5eabf346767589a4f6"
Jul 28 20:13:48.242917 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.242900    2572 scope.go:117] "RemoveContainer" containerID="8b22a2c3a6cbd67e128a89c063e08ceee0b2a87a3301bb5eabf346767589a4f6"
Jul 28 20:13:48.243154 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:13:48.243136    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b22a2c3a6cbd67e128a89c063e08ceee0b2a87a3301bb5eabf346767589a4f6\": container with ID starting with 8b22a2c3a6cbd67e128a89c063e08ceee0b2a87a3301bb5eabf346767589a4f6 not found: ID does not exist" containerID="8b22a2c3a6cbd67e128a89c063e08ceee0b2a87a3301bb5eabf346767589a4f6"
Jul 28 20:13:48.243228 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.243161    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b22a2c3a6cbd67e128a89c063e08ceee0b2a87a3301bb5eabf346767589a4f6"} err="failed to get container status \"8b22a2c3a6cbd67e128a89c063e08ceee0b2a87a3301bb5eabf346767589a4f6\": rpc error: code = NotFound desc = could not find container \"8b22a2c3a6cbd67e128a89c063e08ceee0b2a87a3301bb5eabf346767589a4f6\": container with ID starting with 8b22a2c3a6cbd67e128a89c063e08ceee0b2a87a3301bb5eabf346767589a4f6 not found: ID does not exist"
Jul 28 20:13:48.255887 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.255868    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"]
Jul 28 20:13:48.260776 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.260756    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-68aa77/test-model-68aa77-kserve-ccc86984f-gjt85"]
Jul 28 20:13:48.724595 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.724536    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw" podUID="a0a48b16-f022-4df8-a231-80f281e5ffcc" containerName="main" containerID="cri-o://6669a5e540f63f4657938ed93ad4732f5bcfc695502bebc166f04d9bc8274d70" gracePeriod=45
Jul 28 20:13:48.864549 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.864521    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50760b3f-80de-423f-bb38-2f8f0f404a0b" path="/var/lib/kubelet/pods/50760b3f-80de-423f-bb38-2f8f0f404a0b/volumes"
Jul 28 20:13:48.978437 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:48.978388    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:13:49.023025 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.023001    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-home\") pod \"a0a48b16-f022-4df8-a231-80f281e5ffcc\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") "
Jul 28 20:13:49.023150 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.023060    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-model-cache\") pod \"a0a48b16-f022-4df8-a231-80f281e5ffcc\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") "
Jul 28 20:13:49.023150 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.023116    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-tmp-dir\") pod \"a0a48b16-f022-4df8-a231-80f281e5ffcc\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") "
Jul 28 20:13:49.023150 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.023145    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0a48b16-f022-4df8-a231-80f281e5ffcc-tls-certs\") pod \"a0a48b16-f022-4df8-a231-80f281e5ffcc\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") "
Jul 28 20:13:49.023308 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.023261    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-home" (OuterVolumeSpecName: "home") pod "a0a48b16-f022-4df8-a231-80f281e5ffcc" (UID: "a0a48b16-f022-4df8-a231-80f281e5ffcc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:13:49.023383 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.023359    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a0a48b16-f022-4df8-a231-80f281e5ffcc" (UID: "a0a48b16-f022-4df8-a231-80f281e5ffcc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:13:49.023383 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.023377    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-model-cache" (OuterVolumeSpecName: "model-cache") pod "a0a48b16-f022-4df8-a231-80f281e5ffcc" (UID: "a0a48b16-f022-4df8-a231-80f281e5ffcc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:13:49.023482 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.023395    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78zxr\" (UniqueName: \"kubernetes.io/projected/a0a48b16-f022-4df8-a231-80f281e5ffcc-kube-api-access-78zxr\") pod \"a0a48b16-f022-4df8-a231-80f281e5ffcc\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") "
Jul 28 20:13:49.023482 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.023431    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-dshm\") pod \"a0a48b16-f022-4df8-a231-80f281e5ffcc\" (UID: \"a0a48b16-f022-4df8-a231-80f281e5ffcc\") "
Jul 28 20:13:49.023684 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.023668    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-home\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:13:49.023751 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.023688    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-model-cache\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:13:49.023751 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.023699    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-tmp-dir\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:13:49.025319 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.025299    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0a48b16-f022-4df8-a231-80f281e5ffcc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a0a48b16-f022-4df8-a231-80f281e5ffcc" (UID: "a0a48b16-f022-4df8-a231-80f281e5ffcc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:13:49.025319 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.025309    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-dshm" (OuterVolumeSpecName: "dshm") pod "a0a48b16-f022-4df8-a231-80f281e5ffcc" (UID: "a0a48b16-f022-4df8-a231-80f281e5ffcc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:13:49.025427 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.025404    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0a48b16-f022-4df8-a231-80f281e5ffcc-kube-api-access-78zxr" (OuterVolumeSpecName: "kube-api-access-78zxr") pod "a0a48b16-f022-4df8-a231-80f281e5ffcc" (UID: "a0a48b16-f022-4df8-a231-80f281e5ffcc"). InnerVolumeSpecName "kube-api-access-78zxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:13:49.124644 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.124609    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0a48b16-f022-4df8-a231-80f281e5ffcc-tls-certs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:13:49.124644 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.124643    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78zxr\" (UniqueName: \"kubernetes.io/projected/a0a48b16-f022-4df8-a231-80f281e5ffcc-kube-api-access-78zxr\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:13:49.124644 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.124653    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0a48b16-f022-4df8-a231-80f281e5ffcc-dshm\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:13:49.240882 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.240809    2572 generic.go:356] "Generic (PLEG): container finished" podID="a0a48b16-f022-4df8-a231-80f281e5ffcc" containerID="6669a5e540f63f4657938ed93ad4732f5bcfc695502bebc166f04d9bc8274d70" exitCode=0
Jul 28 20:13:49.240882 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.240852    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"
Jul 28 20:13:49.241029 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.240896    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw" event={"ID":"a0a48b16-f022-4df8-a231-80f281e5ffcc","Type":"ContainerDied","Data":"6669a5e540f63f4657938ed93ad4732f5bcfc695502bebc166f04d9bc8274d70"}
Jul 28 20:13:49.241029 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.240940    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw" event={"ID":"a0a48b16-f022-4df8-a231-80f281e5ffcc","Type":"ContainerDied","Data":"d8c9501247faaf5ec0998beaca7a84084d1dfc7b0dfb4fadbc16554da6c52292"}
Jul 28 20:13:49.241029 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.240959    2572 scope.go:117] "RemoveContainer" containerID="6669a5e540f63f4657938ed93ad4732f5bcfc695502bebc166f04d9bc8274d70"
Jul 28 20:13:49.250856 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.250841    2572 scope.go:117] "RemoveContainer" containerID="6669a5e540f63f4657938ed93ad4732f5bcfc695502bebc166f04d9bc8274d70"
Jul 28 20:13:49.251134 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:13:49.251114    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6669a5e540f63f4657938ed93ad4732f5bcfc695502bebc166f04d9bc8274d70\": container with ID starting with 6669a5e540f63f4657938ed93ad4732f5bcfc695502bebc166f04d9bc8274d70 not found: ID does not exist" containerID="6669a5e540f63f4657938ed93ad4732f5bcfc695502bebc166f04d9bc8274d70"
Jul 28 20:13:49.251186 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.251142    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6669a5e540f63f4657938ed93ad4732f5bcfc695502bebc166f04d9bc8274d70"} err="failed to get container status \"6669a5e540f63f4657938ed93ad4732f5bcfc695502bebc166f04d9bc8274d70\": rpc error: code = NotFound desc = could not find container \"6669a5e540f63f4657938ed93ad4732f5bcfc695502bebc166f04d9bc8274d70\": container with ID starting with 6669a5e540f63f4657938ed93ad4732f5bcfc695502bebc166f04d9bc8274d70 not found: ID does not exist"
Jul 28 20:13:49.261889 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.261866    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"]
Jul 28 20:13:49.268094 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:49.268054    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw"]
Jul 28 20:13:50.822537 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:50.822501    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:50.864296 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:50.864269    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0a48b16-f022-4df8-a231-80f281e5ffcc" path="/var/lib/kubelet/pods/a0a48b16-f022-4df8-a231-80f281e5ffcc/volumes"
Jul 28 20:13:50.921358 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:50.921308    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:51.087879 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:51.087810    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:51.310708 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:51.310659    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:51.417548 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:51.417478    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2"]
Jul 28 20:13:51.417862 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:51.417849    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0a48b16-f022-4df8-a231-80f281e5ffcc" containerName="main"
Jul 28 20:13:51.417902 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:51.417864    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0a48b16-f022-4df8-a231-80f281e5ffcc" containerName="main"
Jul 28 20:13:51.417902 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:51.417887    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50760b3f-80de-423f-bb38-2f8f0f404a0b" containerName="main"
Jul 28 20:13:51.417902 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:51.417893    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="50760b3f-80de-423f-bb38-2f8f0f404a0b" containerName="main"
Jul 28 20:13:51.417998 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:51.417949    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="50760b3f-80de-423f-bb38-2f8f0f404a0b" containerName="main"
Jul 28 20:13:51.417998 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:51.417957    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0a48b16-f022-4df8-a231-80f281e5ffcc" containerName="main"
Jul 28 20:13:51.422418 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:51.422400    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2"
Jul 28 20:13:51.425487 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:51.425463    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6zs4\""
Jul 28 20:13:51.427913 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:51.427888    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2"]
Jul 28 20:13:51.443829 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:51.443798    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/453f888d-7456-47fe-abfb-6e0138c8f08a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2\" (UID: \"453f888d-7456-47fe-abfb-6e0138c8f08a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2"
Jul 28 20:13:51.544697 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:51.544674    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/453f888d-7456-47fe-abfb-6e0138c8f08a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2\" (UID: \"453f888d-7456-47fe-abfb-6e0138c8f08a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2"
Jul 28 20:13:51.545224 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:51.545206    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/453f888d-7456-47fe-abfb-6e0138c8f08a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2\" (UID: \"453f888d-7456-47fe-abfb-6e0138c8f08a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2"
Jul 28 20:13:51.734273 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:51.734207    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2"
Jul 28 20:13:51.860250 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:51.860222    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2"]
Jul 28 20:13:51.861256 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:13:51.861231    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod453f888d_7456_47fe_abfb_6e0138c8f08a.slice/crio-4ac92ac4ee16c781fd766d12c4cdefa89d9c4910a26cf53e5a7aa7446421a628 WatchSource:0}: Error finding container 4ac92ac4ee16c781fd766d12c4cdefa89d9c4910a26cf53e5a7aa7446421a628: Status 404 returned error can't find the container with id 4ac92ac4ee16c781fd766d12c4cdefa89d9c4910a26cf53e5a7aa7446421a628
Jul 28 20:13:52.264353 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:52.264323    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2" event={"ID":"453f888d-7456-47fe-abfb-6e0138c8f08a","Type":"ContainerStarted","Data":"06fe2c3965ee0184ddbec6989e2b3451a21d1394254f87187fb60839fde03e14"}
Jul 28 20:13:52.264456 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:52.264403    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2" event={"ID":"453f888d-7456-47fe-abfb-6e0138c8f08a","Type":"ContainerStarted","Data":"4ac92ac4ee16c781fd766d12c4cdefa89d9c4910a26cf53e5a7aa7446421a628"}
Jul 28 20:13:52.281608 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:52.281566    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2" podStartSLOduration=1.28155116 podStartE2EDuration="1.28155116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:13:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:13:52.278504477 +0000 UTC m=+2657.866167421" watchObservedRunningTime="2026-07-28 20:13:52.28155116 +0000 UTC m=+2657.869214106"
Jul 28 20:13:53.271678 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:53.271640    2572 generic.go:356] "Generic (PLEG): container finished" podID="453f888d-7456-47fe-abfb-6e0138c8f08a" containerID="06fe2c3965ee0184ddbec6989e2b3451a21d1394254f87187fb60839fde03e14" exitCode=0
Jul 28 20:13:53.272060 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:53.271676    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2" event={"ID":"453f888d-7456-47fe-abfb-6e0138c8f08a","Type":"ContainerDied","Data":"06fe2c3965ee0184ddbec6989e2b3451a21d1394254f87187fb60839fde03e14"}
Jul 28 20:13:53.959166 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:53.959129    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-b-e98043/test-model-e98043-kserve-7666d988dc-mxmpw" podUID="a0a48b16-f022-4df8-a231-80f281e5ffcc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 28 20:13:54.408779 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:54.408755    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2"
Jul 28 20:13:54.468980 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:54.468956    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/453f888d-7456-47fe-abfb-6e0138c8f08a-maas-api-service-ca\") pod \"453f888d-7456-47fe-abfb-6e0138c8f08a\" (UID: \"453f888d-7456-47fe-abfb-6e0138c8f08a\") "
Jul 28 20:13:54.469288 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:54.469268    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/453f888d-7456-47fe-abfb-6e0138c8f08a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "453f888d-7456-47fe-abfb-6e0138c8f08a" (UID: "453f888d-7456-47fe-abfb-6e0138c8f08a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:13:54.570236 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:54.570214    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/453f888d-7456-47fe-abfb-6e0138c8f08a-maas-api-service-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:13:55.285598 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:55.285517    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2" event={"ID":"453f888d-7456-47fe-abfb-6e0138c8f08a","Type":"ContainerDied","Data":"4ac92ac4ee16c781fd766d12c4cdefa89d9c4910a26cf53e5a7aa7446421a628"}
Jul 28 20:13:55.285598 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:55.285534    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2"
Jul 28 20:13:55.285598 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:13:55.285573    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ac92ac4ee16c781fd766d12c4cdefa89d9c4910a26cf53e5a7aa7446421a628"
Jul 28 20:14:02.061461 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:02.061420    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2"]
Jul 28 20:14:02.066272 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:02.066249    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e98043-2lsz2"]
Jul 28 20:14:02.165792 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:02.165762    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:02.233861 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:02.233839    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:02.416122 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:02.416044    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:02.863695 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:02.863664    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="453f888d-7456-47fe-abfb-6e0138c8f08a" path="/var/lib/kubelet/pods/453f888d-7456-47fe-abfb-6e0138c8f08a/volumes"
Jul 28 20:14:08.330895 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:08.330862    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:08.913563 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:08.913527    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:08.963856 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:08.963821    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:09.165022 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:09.164951    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:09.465944 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:09.465867    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:09.783913 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:09.783834    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:12.497981 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:12.497948    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:12.579003 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:12.578965    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:12.652124 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:12.652095    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:12.987909 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:12.987878    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:13.279569 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:13.279495    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:13.389815 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:13.389786    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:35.001381 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:35.001312    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 20:14:35.005206 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:35.005180    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 20:14:37.472500 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:37.472462    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:37.806416 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:37.806378    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:37.975674 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:37.975642    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:38.307315 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:38.307289    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:38.449442 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:38.449412    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:41.673050 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:41.673020    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:41.771788 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:41.771736    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:41.888181 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:41.888158    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:42.136585 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:42.136556    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:42.456352 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:42.456282    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:42.563569 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:14:42.563540    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:00.154968 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:00.154926    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-0180d1-29754495-j867g"]
Jul 28 20:15:00.155566 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:00.155541    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="453f888d-7456-47fe-abfb-6e0138c8f08a" containerName="revoke-keys"
Jul 28 20:15:00.155624 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:00.155568    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="453f888d-7456-47fe-abfb-6e0138c8f08a" containerName="revoke-keys"
Jul 28 20:15:00.155709 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:00.155695    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="453f888d-7456-47fe-abfb-6e0138c8f08a" containerName="revoke-keys"
Jul 28 20:15:00.159161 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:00.159140    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754495-n4xv2"]
Jul 28 20:15:00.159285 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:00.159264    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-0180d1-29754495-j867g"
Jul 28 20:15:00.161877 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:00.161852    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6zs4\""
Jul 28 20:15:00.162776 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:00.162567    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754495-n4xv2"
Jul 28 20:15:00.166170 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:00.166145    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754495-n4xv2"]
Jul 28 20:15:00.169490 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:00.169369    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-0180d1-29754495-j867g"]
Jul 28 20:15:00.175886 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:00.175861    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-0180d1-29754495-j867g"
Jul 28 20:15:00.182636 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:00.182616    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754495-n4xv2"
Jul 28 20:15:00.343110 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:00.343087    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754495-n4xv2"]
Jul 28 20:15:00.344802 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:15:00.344773    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb942e1a_4c20_48c4_b5d3_db5227cf3a98.slice/crio-b40cfdfe52f0be1639380e9e16b901e185bc75f6549923260b1e1f60fa26df27 WatchSource:0}: Error finding container b40cfdfe52f0be1639380e9e16b901e185bc75f6549923260b1e1f60fa26df27: Status 404 returned error can't find the container with id b40cfdfe52f0be1639380e9e16b901e185bc75f6549923260b1e1f60fa26df27
Jul 28 20:15:00.520846 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:00.520825    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-0180d1-29754495-j867g"]
Jul 28 20:15:00.522992 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:15:00.522970    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5886d57_bf07_4ef1_a3f2_6d38eeaff3b5.slice/crio-395fb5552dc3db3d6175c88bc5f548774b1395291ed6ba67865ac2f4ce971426 WatchSource:0}: Error finding container 395fb5552dc3db3d6175c88bc5f548774b1395291ed6ba67865ac2f4ce971426: Status 404 returned error can't find the container with id 395fb5552dc3db3d6175c88bc5f548774b1395291ed6ba67865ac2f4ce971426
Jul 28 20:15:00.659627 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:00.659600    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-0180d1-29754495-j867g" event={"ID":"b5886d57-bf07-4ef1-a3f2-6d38eeaff3b5","Type":"ContainerStarted","Data":"395fb5552dc3db3d6175c88bc5f548774b1395291ed6ba67865ac2f4ce971426"}
Jul 28 20:15:00.661100 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:00.661079    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754495-n4xv2" event={"ID":"db942e1a-4c20-48c4-b5d3-db5227cf3a98","Type":"ContainerStarted","Data":"85400e360efe23da52bf06b2423882e060b651b2f3d57430661022d3cfe4da6a"}
Jul 28 20:15:00.661179 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:00.661118    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754495-n4xv2" event={"ID":"db942e1a-4c20-48c4-b5d3-db5227cf3a98","Type":"ContainerStarted","Data":"b40cfdfe52f0be1639380e9e16b901e185bc75f6549923260b1e1f60fa26df27"}
Jul 28 20:15:00.677725 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:00.677687    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754495-n4xv2" podStartSLOduration=0.677673895 podStartE2EDuration="677.673895ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:15:00.676156876 +0000 UTC m=+2726.263819855" watchObservedRunningTime="2026-07-28 20:15:00.677673895 +0000 UTC m=+2726.265336839"
Jul 28 20:15:01.668717 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:01.668683    2572 generic.go:356] "Generic (PLEG): container finished" podID="db942e1a-4c20-48c4-b5d3-db5227cf3a98" containerID="85400e360efe23da52bf06b2423882e060b651b2f3d57430661022d3cfe4da6a" exitCode=0
Jul 28 20:15:01.669114 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:01.668771    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754495-n4xv2" event={"ID":"db942e1a-4c20-48c4-b5d3-db5227cf3a98","Type":"ContainerDied","Data":"85400e360efe23da52bf06b2423882e060b651b2f3d57430661022d3cfe4da6a"}
Jul 28 20:15:01.670476 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:01.670452    2572 generic.go:356] "Generic (PLEG): container finished" podID="b5886d57-bf07-4ef1-a3f2-6d38eeaff3b5" containerID="69ad994285498b60a203921d5be0694a6f80dbfae74d85acd88d8567be365afe" exitCode=0
Jul 28 20:15:01.670579 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:01.670537    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-0180d1-29754495-j867g" event={"ID":"b5886d57-bf07-4ef1-a3f2-6d38eeaff3b5","Type":"ContainerDied","Data":"69ad994285498b60a203921d5be0694a6f80dbfae74d85acd88d8567be365afe"}
Jul 28 20:15:02.819193 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:02.819173    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754495-n4xv2"
Jul 28 20:15:02.834293 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:02.834269    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-0180d1-29754495-j867g"
Jul 28 20:15:03.682442 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:03.682411    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754495-n4xv2"
Jul 28 20:15:03.682597 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:03.682454    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754495-n4xv2" event={"ID":"db942e1a-4c20-48c4-b5d3-db5227cf3a98","Type":"ContainerDied","Data":"b40cfdfe52f0be1639380e9e16b901e185bc75f6549923260b1e1f60fa26df27"}
Jul 28 20:15:03.682597 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:03.682503    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b40cfdfe52f0be1639380e9e16b901e185bc75f6549923260b1e1f60fa26df27"
Jul 28 20:15:03.683972 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:03.683951    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-0180d1-29754495-j867g"
Jul 28 20:15:03.684140 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:03.683992    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-0180d1-29754495-j867g" event={"ID":"b5886d57-bf07-4ef1-a3f2-6d38eeaff3b5","Type":"ContainerDied","Data":"395fb5552dc3db3d6175c88bc5f548774b1395291ed6ba67865ac2f4ce971426"}
Jul 28 20:15:03.684140 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:03.684042    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="395fb5552dc3db3d6175c88bc5f548774b1395291ed6ba67865ac2f4ce971426"
Jul 28 20:15:06.757438 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:06.757405    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:09.502733 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:09.502693    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6"]
Jul 28 20:15:09.503105 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:09.503078    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db942e1a-4c20-48c4-b5d3-db5227cf3a98" containerName="cleanup"
Jul 28 20:15:09.503105 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:09.503099    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="db942e1a-4c20-48c4-b5d3-db5227cf3a98" containerName="cleanup"
Jul 28 20:15:09.503105 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:09.503107    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5886d57-bf07-4ef1-a3f2-6d38eeaff3b5" containerName="cleanup"
Jul 28 20:15:09.503105 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:09.503113    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5886d57-bf07-4ef1-a3f2-6d38eeaff3b5" containerName="cleanup"
Jul 28 20:15:09.503265 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:09.503170    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5886d57-bf07-4ef1-a3f2-6d38eeaff3b5" containerName="cleanup"
Jul 28 20:15:09.503265 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:09.503178    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="db942e1a-4c20-48c4-b5d3-db5227cf3a98" containerName="cleanup"
Jul 28 20:15:09.506309 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:09.506291    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6"
Jul 28 20:15:09.508610 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:09.508588    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6zs4\""
Jul 28 20:15:09.513221 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:09.513194    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6"]
Jul 28 20:15:09.566765 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:09.566743    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf5d73eb-d69d-4560-b0dc-72d48c2dca87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6\" (UID: \"bf5d73eb-d69d-4560-b0dc-72d48c2dca87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6"
Jul 28 20:15:09.667338 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:09.667318    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf5d73eb-d69d-4560-b0dc-72d48c2dca87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6\" (UID: \"bf5d73eb-d69d-4560-b0dc-72d48c2dca87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6"
Jul 28 20:15:09.667877 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:09.667858    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf5d73eb-d69d-4560-b0dc-72d48c2dca87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6\" (UID: \"bf5d73eb-d69d-4560-b0dc-72d48c2dca87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6"
Jul 28 20:15:09.818304 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:09.818284    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6"
Jul 28 20:15:10.153952 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:10.153923    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6"]
Jul 28 20:15:10.155433 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:15:10.155407    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf5d73eb_d69d_4560_b0dc_72d48c2dca87.slice/crio-3df2c4e3820996ee7104c60d896664370e801b3996d6a019eddaf85fc8bf3e35 WatchSource:0}: Error finding container 3df2c4e3820996ee7104c60d896664370e801b3996d6a019eddaf85fc8bf3e35: Status 404 returned error can't find the container with id 3df2c4e3820996ee7104c60d896664370e801b3996d6a019eddaf85fc8bf3e35
Jul 28 20:15:10.724254 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:10.724195    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6" event={"ID":"bf5d73eb-d69d-4560-b0dc-72d48c2dca87","Type":"ContainerStarted","Data":"285829df37947246c465f500086ff53cdd71ba22cbe944d70903f02c5bdcf321"}
Jul 28 20:15:10.724254 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:10.724243    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6" event={"ID":"bf5d73eb-d69d-4560-b0dc-72d48c2dca87","Type":"ContainerStarted","Data":"3df2c4e3820996ee7104c60d896664370e801b3996d6a019eddaf85fc8bf3e35"}
Jul 28 20:15:10.742532 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:10.742491    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6" podStartSLOduration=1.7424775719999999 podStartE2EDuration="1.742477572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:15:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:15:10.739226 +0000 UTC m=+2736.326888942" watchObservedRunningTime="2026-07-28 20:15:10.742477572 +0000 UTC m=+2736.330140517"
Jul 28 20:15:11.732380 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:11.732341    2572 generic.go:356] "Generic (PLEG): container finished" podID="bf5d73eb-d69d-4560-b0dc-72d48c2dca87" containerID="285829df37947246c465f500086ff53cdd71ba22cbe944d70903f02c5bdcf321" exitCode=0
Jul 28 20:15:11.732790 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:11.732426    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6" event={"ID":"bf5d73eb-d69d-4560-b0dc-72d48c2dca87","Type":"ContainerDied","Data":"285829df37947246c465f500086ff53cdd71ba22cbe944d70903f02c5bdcf321"}
Jul 28 20:15:12.883501 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:12.883481    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6"
Jul 28 20:15:13.000517 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:13.000449    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf5d73eb-d69d-4560-b0dc-72d48c2dca87-maas-api-service-ca\") pod \"bf5d73eb-d69d-4560-b0dc-72d48c2dca87\" (UID: \"bf5d73eb-d69d-4560-b0dc-72d48c2dca87\") "
Jul 28 20:15:13.000771 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:13.000749    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf5d73eb-d69d-4560-b0dc-72d48c2dca87-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bf5d73eb-d69d-4560-b0dc-72d48c2dca87" (UID: "bf5d73eb-d69d-4560-b0dc-72d48c2dca87"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:15:13.101381 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:13.101352    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf5d73eb-d69d-4560-b0dc-72d48c2dca87-maas-api-service-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:15:13.749415 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:13.749384    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6" event={"ID":"bf5d73eb-d69d-4560-b0dc-72d48c2dca87","Type":"ContainerDied","Data":"3df2c4e3820996ee7104c60d896664370e801b3996d6a019eddaf85fc8bf3e35"}
Jul 28 20:15:13.749542 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:13.749427    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6"
Jul 28 20:15:13.749542 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:13.749457    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3df2c4e3820996ee7104c60d896664370e801b3996d6a019eddaf85fc8bf3e35"
Jul 28 20:15:25.140951 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:25.140899    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:25.146249 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:25.146223    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-0180d1-29754495-j867g"]
Jul 28 20:15:25.148357 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:25.148337    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-0180d1-29754495-j867g"]
Jul 28 20:15:25.222707 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:25.222680    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:25.306849 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:25.306820    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:25.355049 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:25.355021    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6"]
Jul 28 20:15:25.357109 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:25.357090    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0180d1-2h4t6"]
Jul 28 20:15:25.392582 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:25.392499    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:25.713125 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:25.713019    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:26.864678 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:26.864633    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5886d57-bf07-4ef1-a3f2-6d38eeaff3b5" path="/var/lib/kubelet/pods/b5886d57-bf07-4ef1-a3f2-6d38eeaff3b5/volumes"
Jul 28 20:15:26.865064 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:26.864946    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf5d73eb-d69d-4560-b0dc-72d48c2dca87" path="/var/lib/kubelet/pods/bf5d73eb-d69d-4560-b0dc-72d48c2dca87/volumes"
Jul 28 20:15:31.516743 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:31.516706    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:31.578120 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:31.578088    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:31.786259 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:31.786184    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:32.002574 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:32.002529    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:32.128549 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:32.128516    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g"]
Jul 28 20:15:32.128951 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:32.128933    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf5d73eb-d69d-4560-b0dc-72d48c2dca87" containerName="revoke-keys"
Jul 28 20:15:32.129034 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:32.128953    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf5d73eb-d69d-4560-b0dc-72d48c2dca87" containerName="revoke-keys"
Jul 28 20:15:32.129130 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:32.129045    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf5d73eb-d69d-4560-b0dc-72d48c2dca87" containerName="revoke-keys"
Jul 28 20:15:32.133217 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:32.133197    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g"
Jul 28 20:15:32.135850 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:32.135827    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6zs4\""
Jul 28 20:15:32.139330 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:32.139311    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g"]
Jul 28 20:15:32.258363 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:32.258341    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/874246de-551b-4ee6-8b28-3e61591fab47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g\" (UID: \"874246de-551b-4ee6-8b28-3e61591fab47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g"
Jul 28 20:15:32.358854 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:32.358821    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/874246de-551b-4ee6-8b28-3e61591fab47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g\" (UID: \"874246de-551b-4ee6-8b28-3e61591fab47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g"
Jul 28 20:15:32.359561 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:32.359535    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/874246de-551b-4ee6-8b28-3e61591fab47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g\" (UID: \"874246de-551b-4ee6-8b28-3e61591fab47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g"
Jul 28 20:15:32.444975 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:32.444917    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g"
Jul 28 20:15:32.575652 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:32.575599    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g"]
Jul 28 20:15:32.577730 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:15:32.577705    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod874246de_551b_4ee6_8b28_3e61591fab47.slice/crio-490c4c74b412554434b147302785f6aff7ca32fff728df7bd8ed059e9dd73195 WatchSource:0}: Error finding container 490c4c74b412554434b147302785f6aff7ca32fff728df7bd8ed059e9dd73195: Status 404 returned error can't find the container with id 490c4c74b412554434b147302785f6aff7ca32fff728df7bd8ed059e9dd73195
Jul 28 20:15:32.872295 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:32.872264    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g" event={"ID":"874246de-551b-4ee6-8b28-3e61591fab47","Type":"ContainerStarted","Data":"84245d92883ed826c4aa198d85ac5fd082f761c36d30ed11772b9f4049602b0e"}
Jul 28 20:15:32.872397 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:32.872310    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g" event={"ID":"874246de-551b-4ee6-8b28-3e61591fab47","Type":"ContainerStarted","Data":"490c4c74b412554434b147302785f6aff7ca32fff728df7bd8ed059e9dd73195"}
Jul 28 20:15:32.891153 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:32.891115    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g" podStartSLOduration=0.891101747 podStartE2EDuration="891.101747ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:15:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:15:32.889474362 +0000 UTC m=+2758.477137308" watchObservedRunningTime="2026-07-28 20:15:32.891101747 +0000 UTC m=+2758.478764692"
Jul 28 20:15:33.881062 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:33.881023    2572 generic.go:356] "Generic (PLEG): container finished" podID="874246de-551b-4ee6-8b28-3e61591fab47" containerID="84245d92883ed826c4aa198d85ac5fd082f761c36d30ed11772b9f4049602b0e" exitCode=0
Jul 28 20:15:33.881461 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:33.881108    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g" event={"ID":"874246de-551b-4ee6-8b28-3e61591fab47","Type":"ContainerDied","Data":"84245d92883ed826c4aa198d85ac5fd082f761c36d30ed11772b9f4049602b0e"}
Jul 28 20:15:35.011538 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:35.011510    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g"
Jul 28 20:15:35.180119 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:35.180038    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/874246de-551b-4ee6-8b28-3e61591fab47-maas-api-service-ca\") pod \"874246de-551b-4ee6-8b28-3e61591fab47\" (UID: \"874246de-551b-4ee6-8b28-3e61591fab47\") "
Jul 28 20:15:35.180340 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:35.180320    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/874246de-551b-4ee6-8b28-3e61591fab47-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "874246de-551b-4ee6-8b28-3e61591fab47" (UID: "874246de-551b-4ee6-8b28-3e61591fab47"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:15:35.281427 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:35.281402    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/874246de-551b-4ee6-8b28-3e61591fab47-maas-api-service-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:15:35.893880 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:35.893847    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g"
Jul 28 20:15:35.894045 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:35.893905    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g" event={"ID":"874246de-551b-4ee6-8b28-3e61591fab47","Type":"ContainerDied","Data":"490c4c74b412554434b147302785f6aff7ca32fff728df7bd8ed059e9dd73195"}
Jul 28 20:15:35.894045 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:35.893953    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="490c4c74b412554434b147302785f6aff7ca32fff728df7bd8ed059e9dd73195"
Jul 28 20:15:47.672730 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:47.672691    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:47.767734 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:47.767700    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:47.810167 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:47.810139    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g"]
Jul 28 20:15:47.813511 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:47.813488    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86e5cf-fqq9g"]
Jul 28 20:15:47.829305 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:47.829278    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:48.191049 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:48.191014    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:48.865003 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:48.864969    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="874246de-551b-4ee6-8b28-3e61591fab47" path="/var/lib/kubelet/pods/874246de-551b-4ee6-8b28-3e61591fab47/volumes"
Jul 28 20:15:53.929929 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:53.929891    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:53.991788 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:53.991756    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:54.198816 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:54.198749    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:54.424935 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:54.424886    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:54.760404 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:54.760371    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:55.087932 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:55.087898    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:55.334391 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:55.334360    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:57.993105 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:57.993020    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:58.132473 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:58.132443    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:58.228378 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:58.228351    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:58.434175 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:58.434153    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:58.588408 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:58.588382    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:58.683825 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:15:58.683792    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:22.991642 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:22.991608    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:23.403035 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:23.403005    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:23.593959 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:23.593929    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:23.658741 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:23.658666    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:26.158505 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:26.158469    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:26.255496 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:26.255457    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:26.388768 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:26.388735    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:26.583551 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:26.583511    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:26.856055 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:26.855978    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:26.954605 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:26.954579    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:47.992854 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:47.992812    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:51.162838 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:51.162801    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:53.785451 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.785419    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"]
Jul 28 20:16:53.785868 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.785844    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="874246de-551b-4ee6-8b28-3e61591fab47" containerName="revoke-keys"
Jul 28 20:16:53.785868 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.785857    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="874246de-551b-4ee6-8b28-3e61591fab47" containerName="revoke-keys"
Jul 28 20:16:53.785958 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.785918    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="874246de-551b-4ee6-8b28-3e61591fab47" containerName="revoke-keys"
Jul 28 20:16:53.788845 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.788829    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:53.791451 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.791430    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c38c57\"/\"kube-root-ca.crt\""
Jul 28 20:16:53.792851 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.792829    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c38c57\"/\"openshift-service-ca.crt\""
Jul 28 20:16:53.792851 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.792845    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c38c57\"/\"default-dockercfg-hdtxl\""
Jul 28 20:16:53.793019 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.792859    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c38c57\"/\"auth-test-model-c38c57-kserve-self-signed-certs\""
Jul 28 20:16:53.798212 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.798194    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"]
Jul 28 20:16:53.804963 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.804942    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:53.898783 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.898754    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daf2778a-9ae5-450e-97cd-8001d8c12003-tls-certs\") pod \"auth-test-model-c38c57-kserve-856966ccf-sprhn\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") " pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:53.898783 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.898784    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-tmp-dir\") pod \"auth-test-model-c38c57-kserve-856966ccf-sprhn\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") " pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:53.899019 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.898814    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgftl\" (UniqueName: \"kubernetes.io/projected/daf2778a-9ae5-450e-97cd-8001d8c12003-kube-api-access-jgftl\") pod \"auth-test-model-c38c57-kserve-856966ccf-sprhn\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") " pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:53.899019 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.898869    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-home\") pod \"auth-test-model-c38c57-kserve-856966ccf-sprhn\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") " pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:53.899019 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.898950    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-dshm\") pod \"auth-test-model-c38c57-kserve-856966ccf-sprhn\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") " pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:53.899019 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.898995    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-model-cache\") pod \"auth-test-model-c38c57-kserve-856966ccf-sprhn\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") " pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:53.999774 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.999742    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgftl\" (UniqueName: \"kubernetes.io/projected/daf2778a-9ae5-450e-97cd-8001d8c12003-kube-api-access-jgftl\") pod \"auth-test-model-c38c57-kserve-856966ccf-sprhn\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") " pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:53.999926 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.999780    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-home\") pod \"auth-test-model-c38c57-kserve-856966ccf-sprhn\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") " pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:53.999996 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:53.999976    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-dshm\") pod \"auth-test-model-c38c57-kserve-856966ccf-sprhn\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") " pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:54.000094 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:54.000058    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-model-cache\") pod \"auth-test-model-c38c57-kserve-856966ccf-sprhn\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") " pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:54.000165 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:54.000152    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daf2778a-9ae5-450e-97cd-8001d8c12003-tls-certs\") pod \"auth-test-model-c38c57-kserve-856966ccf-sprhn\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") " pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:54.000213 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:54.000184    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-home\") pod \"auth-test-model-c38c57-kserve-856966ccf-sprhn\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") " pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:54.000262 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:54.000222    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-tmp-dir\") pod \"auth-test-model-c38c57-kserve-856966ccf-sprhn\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") " pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:54.000434 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:54.000415    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-model-cache\") pod \"auth-test-model-c38c57-kserve-856966ccf-sprhn\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") " pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:54.000492 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:54.000457    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-tmp-dir\") pod \"auth-test-model-c38c57-kserve-856966ccf-sprhn\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") " pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:54.002276 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:54.002255    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-dshm\") pod \"auth-test-model-c38c57-kserve-856966ccf-sprhn\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") " pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:54.002510 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:54.002495    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daf2778a-9ae5-450e-97cd-8001d8c12003-tls-certs\") pod \"auth-test-model-c38c57-kserve-856966ccf-sprhn\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") " pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:54.007704 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:54.007678    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgftl\" (UniqueName: \"kubernetes.io/projected/daf2778a-9ae5-450e-97cd-8001d8c12003-kube-api-access-jgftl\") pod \"auth-test-model-c38c57-kserve-856966ccf-sprhn\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") " pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:54.098923 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:54.098897    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:16:54.232607 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:54.232584    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"]
Jul 28 20:16:54.235058 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:16:54.235021    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddaf2778a_9ae5_450e_97cd_8001d8c12003.slice/crio-e393d6b9359957891ac1586b3b2d953e8e4ce4b0f3597c59ac15f51338bcbcb4 WatchSource:0}: Error finding container e393d6b9359957891ac1586b3b2d953e8e4ce4b0f3597c59ac15f51338bcbcb4: Status 404 returned error can't find the container with id e393d6b9359957891ac1586b3b2d953e8e4ce4b0f3597c59ac15f51338bcbcb4
Jul 28 20:16:54.310147 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:54.310111    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn" event={"ID":"daf2778a-9ae5-450e-97cd-8001d8c12003","Type":"ContainerStarted","Data":"a794478fa16f00c880802b3237ea2445f9dbbff31879a2ab6c990c47a32a88bd"}
Jul 28 20:16:54.310285 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:54.310165    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn" event={"ID":"daf2778a-9ae5-450e-97cd-8001d8c12003","Type":"ContainerStarted","Data":"e393d6b9359957891ac1586b3b2d953e8e4ce4b0f3597c59ac15f51338bcbcb4"}
Jul 28 20:16:54.327219 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:16:54.327177    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn" podStartSLOduration=1.327162415 podStartE2EDuration="1.327162415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:16:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:16:54.325913821 +0000 UTC m=+2839.913576767" watchObservedRunningTime="2026-07-28 20:16:54.327162415 +0000 UTC m=+2839.914825408"
Jul 28 20:17:04.099551 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:17:04.099512    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:17:04.099934 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:17:04.099563    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:17:04.111763 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:17:04.111737    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:17:04.373353 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:17:04.373289    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:17:41.614499 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:17:41.614422    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:35.892041 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:18:35.892010    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:19:22.474947 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:19:22.474908    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:19:35.039031 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:19:35.039006    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 20:19:35.045609 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:19:35.045589    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 20:19:58.758476 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:19:58.758435    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:19:58.858606 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:19:58.858557    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:20:35.131897 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:20:35.131829    2572 scope.go:117] "RemoveContainer" containerID="06fe2c3965ee0184ddbec6989e2b3451a21d1394254f87187fb60839fde03e14"
Jul 28 20:21:03.054166 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:21:03.054136    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:35.152492 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:21:35.152463    2572 scope.go:117] "RemoveContainer" containerID="285829df37947246c465f500086ff53cdd71ba22cbe944d70903f02c5bdcf321"
Jul 28 20:21:35.160422 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:21:35.160405    2572 scope.go:117] "RemoveContainer" containerID="84245d92883ed826c4aa198d85ac5fd082f761c36d30ed11772b9f4049602b0e"
Jul 28 20:21:35.167581 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:21:35.167560    2572 scope.go:117] "RemoveContainer" containerID="69ad994285498b60a203921d5be0694a6f80dbfae74d85acd88d8567be365afe"
Jul 28 20:22:30.270385 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:30.270345    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"]
Jul 28 20:22:30.451166 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:30.451130    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:31.193237 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:31.193201    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:40.405478 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.405450    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"]
Jul 28 20:22:40.408930 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.408913    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.411398 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.411379    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c38c57\"/\"sub-test-model-c38c57-kserve-self-signed-certs\""
Jul 28 20:22:40.418258 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.418229    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:40.418435 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.418418    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"]
Jul 28 20:22:40.437830 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.437801    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-dshm\") pod \"sub-test-model-c38c57-kserve-94f55585-d9j6z\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") " pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.437943 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.437848    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-home\") pod \"sub-test-model-c38c57-kserve-94f55585-d9j6z\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") " pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.437943 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.437872    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-model-cache\") pod \"sub-test-model-c38c57-kserve-94f55585-d9j6z\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") " pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.437943 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.437932    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mvc7\" (UniqueName: \"kubernetes.io/projected/43d62f69-3682-473f-842d-bdbd6a2005b1-kube-api-access-7mvc7\") pod \"sub-test-model-c38c57-kserve-94f55585-d9j6z\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") " pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.438059 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.437995    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-tmp-dir\") pod \"sub-test-model-c38c57-kserve-94f55585-d9j6z\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") " pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.438128 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.438091    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43d62f69-3682-473f-842d-bdbd6a2005b1-tls-certs\") pod \"sub-test-model-c38c57-kserve-94f55585-d9j6z\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") " pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.539175 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.539111    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mvc7\" (UniqueName: \"kubernetes.io/projected/43d62f69-3682-473f-842d-bdbd6a2005b1-kube-api-access-7mvc7\") pod \"sub-test-model-c38c57-kserve-94f55585-d9j6z\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") " pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.539175 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.539154    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-tmp-dir\") pod \"sub-test-model-c38c57-kserve-94f55585-d9j6z\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") " pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.539362 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.539340    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43d62f69-3682-473f-842d-bdbd6a2005b1-tls-certs\") pod \"sub-test-model-c38c57-kserve-94f55585-d9j6z\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") " pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.539406 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.539395    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-dshm\") pod \"sub-test-model-c38c57-kserve-94f55585-d9j6z\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") " pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.539446 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.539434    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-home\") pod \"sub-test-model-c38c57-kserve-94f55585-d9j6z\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") " pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.539488 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.539459    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-model-cache\") pod \"sub-test-model-c38c57-kserve-94f55585-d9j6z\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") " pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.539588 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.539571    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-tmp-dir\") pod \"sub-test-model-c38c57-kserve-94f55585-d9j6z\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") " pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.539737 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.539707    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-home\") pod \"sub-test-model-c38c57-kserve-94f55585-d9j6z\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") " pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.539838 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.539802    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-model-cache\") pod \"sub-test-model-c38c57-kserve-94f55585-d9j6z\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") " pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.541572 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.541554    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-dshm\") pod \"sub-test-model-c38c57-kserve-94f55585-d9j6z\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") " pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.541906 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.541891    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43d62f69-3682-473f-842d-bdbd6a2005b1-tls-certs\") pod \"sub-test-model-c38c57-kserve-94f55585-d9j6z\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") " pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.549729 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.549708    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mvc7\" (UniqueName: \"kubernetes.io/projected/43d62f69-3682-473f-842d-bdbd6a2005b1-kube-api-access-7mvc7\") pod \"sub-test-model-c38c57-kserve-94f55585-d9j6z\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") " pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.719687 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.719656    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:40.852213 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:40.852181    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"]
Jul 28 20:22:40.853623 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:22:40.853593    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43d62f69_3682_473f_842d_bdbd6a2005b1.slice/crio-295a1cd26cfdb34467c549f7dc7990814bab08c93bbc54c5688f2830358000c3 WatchSource:0}: Error finding container 295a1cd26cfdb34467c549f7dc7990814bab08c93bbc54c5688f2830358000c3: Status 404 returned error can't find the container with id 295a1cd26cfdb34467c549f7dc7990814bab08c93bbc54c5688f2830358000c3
Jul 28 20:22:41.177538 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:41.177501    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z" event={"ID":"43d62f69-3682-473f-842d-bdbd6a2005b1","Type":"ContainerStarted","Data":"68d9537e3aaa3eb022c62738a82c8d7f6acddebc58b9c94cfd4695e8f0d0415d"}
Jul 28 20:22:41.177538 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:41.177539    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z" event={"ID":"43d62f69-3682-473f-842d-bdbd6a2005b1","Type":"ContainerStarted","Data":"295a1cd26cfdb34467c549f7dc7990814bab08c93bbc54c5688f2830358000c3"}
Jul 28 20:22:41.195491 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:41.195439    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z" podStartSLOduration=1.195424724 podStartE2EDuration="1.195424724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:22:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:22:41.192887563 +0000 UTC m=+3186.780550508" watchObservedRunningTime="2026-07-28 20:22:41.195424724 +0000 UTC m=+3186.783087670"
Jul 28 20:22:45.289056 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.289015    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn" podUID="daf2778a-9ae5-450e-97cd-8001d8c12003" containerName="main" containerID="cri-o://a794478fa16f00c880802b3237ea2445f9dbbff31879a2ab6c990c47a32a88bd" gracePeriod=45
Jul 28 20:22:45.543446 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.543386    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:22:45.582482 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.582453    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgftl\" (UniqueName: \"kubernetes.io/projected/daf2778a-9ae5-450e-97cd-8001d8c12003-kube-api-access-jgftl\") pod \"daf2778a-9ae5-450e-97cd-8001d8c12003\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") "
Jul 28 20:22:45.582663 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.582509    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-tmp-dir\") pod \"daf2778a-9ae5-450e-97cd-8001d8c12003\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") "
Jul 28 20:22:45.582663 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.582533    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-home\") pod \"daf2778a-9ae5-450e-97cd-8001d8c12003\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") "
Jul 28 20:22:45.582663 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.582580    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-dshm\") pod \"daf2778a-9ae5-450e-97cd-8001d8c12003\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") "
Jul 28 20:22:45.582663 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.582601    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-model-cache\") pod \"daf2778a-9ae5-450e-97cd-8001d8c12003\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") "
Jul 28 20:22:45.582663 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.582625    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daf2778a-9ae5-450e-97cd-8001d8c12003-tls-certs\") pod \"daf2778a-9ae5-450e-97cd-8001d8c12003\" (UID: \"daf2778a-9ae5-450e-97cd-8001d8c12003\") "
Jul 28 20:22:45.582931 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.582888    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-home" (OuterVolumeSpecName: "home") pod "daf2778a-9ae5-450e-97cd-8001d8c12003" (UID: "daf2778a-9ae5-450e-97cd-8001d8c12003"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:22:45.582931 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.582897    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "daf2778a-9ae5-450e-97cd-8001d8c12003" (UID: "daf2778a-9ae5-450e-97cd-8001d8c12003"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:22:45.583039 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.582919    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-model-cache" (OuterVolumeSpecName: "model-cache") pod "daf2778a-9ae5-450e-97cd-8001d8c12003" (UID: "daf2778a-9ae5-450e-97cd-8001d8c12003"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:22:45.584884 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.584852    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-dshm" (OuterVolumeSpecName: "dshm") pod "daf2778a-9ae5-450e-97cd-8001d8c12003" (UID: "daf2778a-9ae5-450e-97cd-8001d8c12003"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:22:45.585089 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.585040    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/daf2778a-9ae5-450e-97cd-8001d8c12003-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "daf2778a-9ae5-450e-97cd-8001d8c12003" (UID: "daf2778a-9ae5-450e-97cd-8001d8c12003"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:22:45.585212 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.585181    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daf2778a-9ae5-450e-97cd-8001d8c12003-kube-api-access-jgftl" (OuterVolumeSpecName: "kube-api-access-jgftl") pod "daf2778a-9ae5-450e-97cd-8001d8c12003" (UID: "daf2778a-9ae5-450e-97cd-8001d8c12003"). InnerVolumeSpecName "kube-api-access-jgftl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:22:45.683950 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.683926    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daf2778a-9ae5-450e-97cd-8001d8c12003-tls-certs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:22:45.683950 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.683947    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgftl\" (UniqueName: \"kubernetes.io/projected/daf2778a-9ae5-450e-97cd-8001d8c12003-kube-api-access-jgftl\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:22:45.683950 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.683956    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-tmp-dir\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:22:45.684178 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.683965    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-home\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:22:45.684178 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.683973    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-dshm\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:22:45.684178 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:45.683982    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daf2778a-9ae5-450e-97cd-8001d8c12003-model-cache\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:22:46.211651 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:46.211610    2572 generic.go:356] "Generic (PLEG): container finished" podID="daf2778a-9ae5-450e-97cd-8001d8c12003" containerID="a794478fa16f00c880802b3237ea2445f9dbbff31879a2ab6c990c47a32a88bd" exitCode=0
Jul 28 20:22:46.211858 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:46.211662    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"
Jul 28 20:22:46.211858 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:46.211694    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn" event={"ID":"daf2778a-9ae5-450e-97cd-8001d8c12003","Type":"ContainerDied","Data":"a794478fa16f00c880802b3237ea2445f9dbbff31879a2ab6c990c47a32a88bd"}
Jul 28 20:22:46.211858 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:46.211731    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn" event={"ID":"daf2778a-9ae5-450e-97cd-8001d8c12003","Type":"ContainerDied","Data":"e393d6b9359957891ac1586b3b2d953e8e4ce4b0f3597c59ac15f51338bcbcb4"}
Jul 28 20:22:46.211858 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:46.211748    2572 scope.go:117] "RemoveContainer" containerID="a794478fa16f00c880802b3237ea2445f9dbbff31879a2ab6c990c47a32a88bd"
Jul 28 20:22:46.221003 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:46.220981    2572 scope.go:117] "RemoveContainer" containerID="a794478fa16f00c880802b3237ea2445f9dbbff31879a2ab6c990c47a32a88bd"
Jul 28 20:22:46.221263 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:22:46.221245    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a794478fa16f00c880802b3237ea2445f9dbbff31879a2ab6c990c47a32a88bd\": container with ID starting with a794478fa16f00c880802b3237ea2445f9dbbff31879a2ab6c990c47a32a88bd not found: ID does not exist" containerID="a794478fa16f00c880802b3237ea2445f9dbbff31879a2ab6c990c47a32a88bd"
Jul 28 20:22:46.221302 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:46.221273    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a794478fa16f00c880802b3237ea2445f9dbbff31879a2ab6c990c47a32a88bd"} err="failed to get container status \"a794478fa16f00c880802b3237ea2445f9dbbff31879a2ab6c990c47a32a88bd\": rpc error: code = NotFound desc = could not find container \"a794478fa16f00c880802b3237ea2445f9dbbff31879a2ab6c990c47a32a88bd\": container with ID starting with a794478fa16f00c880802b3237ea2445f9dbbff31879a2ab6c990c47a32a88bd not found: ID does not exist"
Jul 28 20:22:46.233476 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:46.233457    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"]
Jul 28 20:22:46.236683 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:46.236659    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c38c57/auth-test-model-c38c57-kserve-856966ccf-sprhn"]
Jul 28 20:22:46.864577 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:46.864548    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daf2778a-9ae5-450e-97cd-8001d8c12003" path="/var/lib/kubelet/pods/daf2778a-9ae5-450e-97cd-8001d8c12003/volumes"
Jul 28 20:22:50.720621 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:50.720588    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:50.721022 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:50.720636    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:50.733194 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:50.733165    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:51.250143 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:51.250116    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:22:54.793765 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.793732    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"]
Jul 28 20:22:54.794236 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.794219    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daf2778a-9ae5-450e-97cd-8001d8c12003" containerName="main"
Jul 28 20:22:54.794236 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.794237    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="daf2778a-9ae5-450e-97cd-8001d8c12003" containerName="main"
Jul 28 20:22:54.794324 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.794303    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="daf2778a-9ae5-450e-97cd-8001d8c12003" containerName="main"
Jul 28 20:22:54.797358 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.797343    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:54.801210 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.801188    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d170bd\"/\"kube-root-ca.crt\""
Jul 28 20:22:54.803313 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.803293    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d170bd\"/\"default-dockercfg-zp99p\""
Jul 28 20:22:54.803644 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.803619    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d170bd\"/\"sub-test-model-d170bd-kserve-self-signed-certs\""
Jul 28 20:22:54.804016 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.803296    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d170bd\"/\"openshift-service-ca.crt\""
Jul 28 20:22:54.807722 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.807701    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"]
Jul 28 20:22:54.823889 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.823861    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:54.865595 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.865568    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-home\") pod \"sub-test-model-d170bd-kserve-8697486bc5-4tgpp\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") " pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:54.865691 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.865601    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-model-cache\") pod \"sub-test-model-d170bd-kserve-8697486bc5-4tgpp\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") " pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:54.865691 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.865625    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-tmp-dir\") pod \"sub-test-model-d170bd-kserve-8697486bc5-4tgpp\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") " pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:54.865691 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.865653    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22vhj\" (UniqueName: \"kubernetes.io/projected/169c5629-e446-4612-b1fc-27b269508c4d-kube-api-access-22vhj\") pod \"sub-test-model-d170bd-kserve-8697486bc5-4tgpp\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") " pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:54.865879 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.865726    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-dshm\") pod \"sub-test-model-d170bd-kserve-8697486bc5-4tgpp\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") " pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:54.865879 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.865757    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/169c5629-e446-4612-b1fc-27b269508c4d-tls-certs\") pod \"sub-test-model-d170bd-kserve-8697486bc5-4tgpp\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") " pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:54.967028 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.967001    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-home\") pod \"sub-test-model-d170bd-kserve-8697486bc5-4tgpp\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") " pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:54.967148 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.967030    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-model-cache\") pod \"sub-test-model-d170bd-kserve-8697486bc5-4tgpp\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") " pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:54.967148 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.967050    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-tmp-dir\") pod \"sub-test-model-d170bd-kserve-8697486bc5-4tgpp\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") " pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:54.967148 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.967088    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22vhj\" (UniqueName: \"kubernetes.io/projected/169c5629-e446-4612-b1fc-27b269508c4d-kube-api-access-22vhj\") pod \"sub-test-model-d170bd-kserve-8697486bc5-4tgpp\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") " pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:54.967148 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.967141    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-dshm\") pod \"sub-test-model-d170bd-kserve-8697486bc5-4tgpp\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") " pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:54.967352 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.967168    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/169c5629-e446-4612-b1fc-27b269508c4d-tls-certs\") pod \"sub-test-model-d170bd-kserve-8697486bc5-4tgpp\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") " pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:54.967466 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.967444    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-model-cache\") pod \"sub-test-model-d170bd-kserve-8697486bc5-4tgpp\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") " pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:54.967541 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.967446    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-home\") pod \"sub-test-model-d170bd-kserve-8697486bc5-4tgpp\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") " pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:54.967631 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.967613    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-tmp-dir\") pod \"sub-test-model-d170bd-kserve-8697486bc5-4tgpp\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") " pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:54.969439 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.969417    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-dshm\") pod \"sub-test-model-d170bd-kserve-8697486bc5-4tgpp\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") " pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:54.969596 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.969580    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/169c5629-e446-4612-b1fc-27b269508c4d-tls-certs\") pod \"sub-test-model-d170bd-kserve-8697486bc5-4tgpp\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") " pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:54.976033 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:54.976016    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22vhj\" (UniqueName: \"kubernetes.io/projected/169c5629-e446-4612-b1fc-27b269508c4d-kube-api-access-22vhj\") pod \"sub-test-model-d170bd-kserve-8697486bc5-4tgpp\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") " pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:55.107894 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:55.107865    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:22:55.247264 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:55.247239    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"]
Jul 28 20:22:55.249961 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:22:55.249933    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod169c5629_e446_4612_b1fc_27b269508c4d.slice/crio-e9b9924ac4851379532b737afd3a8c412f0ffb6bff2818e5a1230bf1365b4797 WatchSource:0}: Error finding container e9b9924ac4851379532b737afd3a8c412f0ffb6bff2818e5a1230bf1365b4797: Status 404 returned error can't find the container with id e9b9924ac4851379532b737afd3a8c412f0ffb6bff2818e5a1230bf1365b4797
Jul 28 20:22:55.261555 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:55.261531    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp" event={"ID":"169c5629-e446-4612-b1fc-27b269508c4d","Type":"ContainerStarted","Data":"e9b9924ac4851379532b737afd3a8c412f0ffb6bff2818e5a1230bf1365b4797"}
Jul 28 20:22:56.268043 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:56.268008    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp" event={"ID":"169c5629-e446-4612-b1fc-27b269508c4d","Type":"ContainerStarted","Data":"93857e902c71602b2687958f62ffb78e7a0672d1d878139df9e39e8638e3a7ad"}
Jul 28 20:22:56.285320 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:22:56.285275    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp" podStartSLOduration=2.285259754 podStartE2EDuration="2.285259754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:22:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:22:56.283476744 +0000 UTC m=+3201.871139692" watchObservedRunningTime="2026-07-28 20:22:56.285259754 +0000 UTC m=+3201.872922700"
Jul 28 20:23:05.108125 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:23:05.108060    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:23:05.108735 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:23:05.108140    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:23:05.120301 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:23:05.120270    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:23:05.329954 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:23:05.329923    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:24:20.382691 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:20.382652    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"]
Jul 28 20:24:21.225220 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:21.225186    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:24:21.309723 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:21.309690    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"]
Jul 28 20:24:22.391234 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:22.391193    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:24:24.542246 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:24.542204    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:24:32.131822 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:32.131787    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:24:35.074166 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.074138    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 20:24:35.082034 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.082010    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 20:24:35.401238 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.401139    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z" podUID="43d62f69-3682-473f-842d-bdbd6a2005b1" containerName="main" containerID="cri-o://68d9537e3aaa3eb022c62738a82c8d7f6acddebc58b9c94cfd4695e8f0d0415d" gracePeriod=45
Jul 28 20:24:35.643602 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.643579    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:24:35.689694 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.689614    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-tmp-dir\") pod \"43d62f69-3682-473f-842d-bdbd6a2005b1\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") "
Jul 28 20:24:35.689694 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.689676    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43d62f69-3682-473f-842d-bdbd6a2005b1-tls-certs\") pod \"43d62f69-3682-473f-842d-bdbd6a2005b1\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") "
Jul 28 20:24:35.689840 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.689764    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-home\") pod \"43d62f69-3682-473f-842d-bdbd6a2005b1\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") "
Jul 28 20:24:35.689840 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.689816    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-model-cache\") pod \"43d62f69-3682-473f-842d-bdbd6a2005b1\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") "
Jul 28 20:24:35.689947 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.689846    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mvc7\" (UniqueName: \"kubernetes.io/projected/43d62f69-3682-473f-842d-bdbd6a2005b1-kube-api-access-7mvc7\") pod \"43d62f69-3682-473f-842d-bdbd6a2005b1\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") "
Jul 28 20:24:35.689947 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.689849    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "43d62f69-3682-473f-842d-bdbd6a2005b1" (UID: "43d62f69-3682-473f-842d-bdbd6a2005b1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:24:35.690061 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.689880    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-dshm\") pod \"43d62f69-3682-473f-842d-bdbd6a2005b1\" (UID: \"43d62f69-3682-473f-842d-bdbd6a2005b1\") "
Jul 28 20:24:35.690148 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.690100    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-home" (OuterVolumeSpecName: "home") pod "43d62f69-3682-473f-842d-bdbd6a2005b1" (UID: "43d62f69-3682-473f-842d-bdbd6a2005b1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:24:35.690202 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.690150    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-model-cache" (OuterVolumeSpecName: "model-cache") pod "43d62f69-3682-473f-842d-bdbd6a2005b1" (UID: "43d62f69-3682-473f-842d-bdbd6a2005b1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:24:35.690583 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.690561    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-tmp-dir\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:24:35.690703 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.690587    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-home\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:24:35.690703 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.690602    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-model-cache\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:24:35.691816 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.691791    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43d62f69-3682-473f-842d-bdbd6a2005b1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "43d62f69-3682-473f-842d-bdbd6a2005b1" (UID: "43d62f69-3682-473f-842d-bdbd6a2005b1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:24:35.691933 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.691900    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-dshm" (OuterVolumeSpecName: "dshm") pod "43d62f69-3682-473f-842d-bdbd6a2005b1" (UID: "43d62f69-3682-473f-842d-bdbd6a2005b1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:24:35.692059 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.692035    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43d62f69-3682-473f-842d-bdbd6a2005b1-kube-api-access-7mvc7" (OuterVolumeSpecName: "kube-api-access-7mvc7") pod "43d62f69-3682-473f-842d-bdbd6a2005b1" (UID: "43d62f69-3682-473f-842d-bdbd6a2005b1"). InnerVolumeSpecName "kube-api-access-7mvc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:24:35.791867 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.791844    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mvc7\" (UniqueName: \"kubernetes.io/projected/43d62f69-3682-473f-842d-bdbd6a2005b1-kube-api-access-7mvc7\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:24:35.791968 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.791871    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43d62f69-3682-473f-842d-bdbd6a2005b1-dshm\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:24:35.791968 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.791887    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43d62f69-3682-473f-842d-bdbd6a2005b1-tls-certs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:24:35.828806 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.828783    2572 generic.go:356] "Generic (PLEG): container finished" podID="43d62f69-3682-473f-842d-bdbd6a2005b1" containerID="68d9537e3aaa3eb022c62738a82c8d7f6acddebc58b9c94cfd4695e8f0d0415d" exitCode=0
Jul 28 20:24:35.828905 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.828830    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"
Jul 28 20:24:35.828905 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.828846    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z" event={"ID":"43d62f69-3682-473f-842d-bdbd6a2005b1","Type":"ContainerDied","Data":"68d9537e3aaa3eb022c62738a82c8d7f6acddebc58b9c94cfd4695e8f0d0415d"}
Jul 28 20:24:35.828905 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.828880    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z" event={"ID":"43d62f69-3682-473f-842d-bdbd6a2005b1","Type":"ContainerDied","Data":"295a1cd26cfdb34467c549f7dc7990814bab08c93bbc54c5688f2830358000c3"}
Jul 28 20:24:35.828905 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.828900    2572 scope.go:117] "RemoveContainer" containerID="68d9537e3aaa3eb022c62738a82c8d7f6acddebc58b9c94cfd4695e8f0d0415d"
Jul 28 20:24:35.840862 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.839796    2572 scope.go:117] "RemoveContainer" containerID="68d9537e3aaa3eb022c62738a82c8d7f6acddebc58b9c94cfd4695e8f0d0415d"
Jul 28 20:24:35.841272 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:24:35.841242    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68d9537e3aaa3eb022c62738a82c8d7f6acddebc58b9c94cfd4695e8f0d0415d\": container with ID starting with 68d9537e3aaa3eb022c62738a82c8d7f6acddebc58b9c94cfd4695e8f0d0415d not found: ID does not exist" containerID="68d9537e3aaa3eb022c62738a82c8d7f6acddebc58b9c94cfd4695e8f0d0415d"
Jul 28 20:24:35.841366 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.841278    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68d9537e3aaa3eb022c62738a82c8d7f6acddebc58b9c94cfd4695e8f0d0415d"} err="failed to get container status \"68d9537e3aaa3eb022c62738a82c8d7f6acddebc58b9c94cfd4695e8f0d0415d\": rpc error: code = NotFound desc = could not find container \"68d9537e3aaa3eb022c62738a82c8d7f6acddebc58b9c94cfd4695e8f0d0415d\": container with ID starting with 68d9537e3aaa3eb022c62738a82c8d7f6acddebc58b9c94cfd4695e8f0d0415d not found: ID does not exist"
Jul 28 20:24:35.853048 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.853027    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"]
Jul 28 20:24:35.863261 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:35.863240    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c38c57/sub-test-model-c38c57-kserve-94f55585-d9j6z"]
Jul 28 20:24:36.328409 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.328375    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp" podUID="169c5629-e446-4612-b1fc-27b269508c4d" containerName="main" containerID="cri-o://93857e902c71602b2687958f62ffb78e7a0672d1d878139df9e39e8638e3a7ad" gracePeriod=45
Jul 28 20:24:36.577415 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.577393    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:24:36.698744 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.698688    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-model-cache\") pod \"169c5629-e446-4612-b1fc-27b269508c4d\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") "
Jul 28 20:24:36.698744 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.698718    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22vhj\" (UniqueName: \"kubernetes.io/projected/169c5629-e446-4612-b1fc-27b269508c4d-kube-api-access-22vhj\") pod \"169c5629-e446-4612-b1fc-27b269508c4d\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") "
Jul 28 20:24:36.698744 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.698736    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/169c5629-e446-4612-b1fc-27b269508c4d-tls-certs\") pod \"169c5629-e446-4612-b1fc-27b269508c4d\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") "
Jul 28 20:24:36.698998 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.698876    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-dshm\") pod \"169c5629-e446-4612-b1fc-27b269508c4d\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") "
Jul 28 20:24:36.698998 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.698928    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-tmp-dir\") pod \"169c5629-e446-4612-b1fc-27b269508c4d\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") "
Jul 28 20:24:36.698998 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.698938    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-model-cache" (OuterVolumeSpecName: "model-cache") pod "169c5629-e446-4612-b1fc-27b269508c4d" (UID: "169c5629-e446-4612-b1fc-27b269508c4d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:24:36.698998 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.698978    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-home\") pod \"169c5629-e446-4612-b1fc-27b269508c4d\" (UID: \"169c5629-e446-4612-b1fc-27b269508c4d\") "
Jul 28 20:24:36.699253 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.699221    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-model-cache\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:24:36.699310 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.699248    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "169c5629-e446-4612-b1fc-27b269508c4d" (UID: "169c5629-e446-4612-b1fc-27b269508c4d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:24:36.699310 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.699260    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-home" (OuterVolumeSpecName: "home") pod "169c5629-e446-4612-b1fc-27b269508c4d" (UID: "169c5629-e446-4612-b1fc-27b269508c4d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:24:36.700948 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.700927    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/169c5629-e446-4612-b1fc-27b269508c4d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "169c5629-e446-4612-b1fc-27b269508c4d" (UID: "169c5629-e446-4612-b1fc-27b269508c4d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:24:36.700948 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.700934    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-dshm" (OuterVolumeSpecName: "dshm") pod "169c5629-e446-4612-b1fc-27b269508c4d" (UID: "169c5629-e446-4612-b1fc-27b269508c4d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:24:36.701087 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.700983    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/169c5629-e446-4612-b1fc-27b269508c4d-kube-api-access-22vhj" (OuterVolumeSpecName: "kube-api-access-22vhj") pod "169c5629-e446-4612-b1fc-27b269508c4d" (UID: "169c5629-e446-4612-b1fc-27b269508c4d"). InnerVolumeSpecName "kube-api-access-22vhj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:24:36.799915 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.799893    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-home\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:24:36.799915 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.799915    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22vhj\" (UniqueName: \"kubernetes.io/projected/169c5629-e446-4612-b1fc-27b269508c4d-kube-api-access-22vhj\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:24:36.800039 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.799926    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/169c5629-e446-4612-b1fc-27b269508c4d-tls-certs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:24:36.800039 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.799936    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-dshm\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:24:36.800039 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.799944    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/169c5629-e446-4612-b1fc-27b269508c4d-tmp-dir\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:24:36.836076 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.836050    2572 generic.go:356] "Generic (PLEG): container finished" podID="169c5629-e446-4612-b1fc-27b269508c4d" containerID="93857e902c71602b2687958f62ffb78e7a0672d1d878139df9e39e8638e3a7ad" exitCode=0
Jul 28 20:24:36.836170 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.836109    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"
Jul 28 20:24:36.836170 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.836135    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp" event={"ID":"169c5629-e446-4612-b1fc-27b269508c4d","Type":"ContainerDied","Data":"93857e902c71602b2687958f62ffb78e7a0672d1d878139df9e39e8638e3a7ad"}
Jul 28 20:24:36.836256 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.836176    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp" event={"ID":"169c5629-e446-4612-b1fc-27b269508c4d","Type":"ContainerDied","Data":"e9b9924ac4851379532b737afd3a8c412f0ffb6bff2818e5a1230bf1365b4797"}
Jul 28 20:24:36.836256 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.836194    2572 scope.go:117] "RemoveContainer" containerID="93857e902c71602b2687958f62ffb78e7a0672d1d878139df9e39e8638e3a7ad"
Jul 28 20:24:36.845345 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.845330    2572 scope.go:117] "RemoveContainer" containerID="93857e902c71602b2687958f62ffb78e7a0672d1d878139df9e39e8638e3a7ad"
Jul 28 20:24:36.845585 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:24:36.845566    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93857e902c71602b2687958f62ffb78e7a0672d1d878139df9e39e8638e3a7ad\": container with ID starting with 93857e902c71602b2687958f62ffb78e7a0672d1d878139df9e39e8638e3a7ad not found: ID does not exist" containerID="93857e902c71602b2687958f62ffb78e7a0672d1d878139df9e39e8638e3a7ad"
Jul 28 20:24:36.845650 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.845592    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93857e902c71602b2687958f62ffb78e7a0672d1d878139df9e39e8638e3a7ad"} err="failed to get container status \"93857e902c71602b2687958f62ffb78e7a0672d1d878139df9e39e8638e3a7ad\": rpc error: code = NotFound desc = could not find container \"93857e902c71602b2687958f62ffb78e7a0672d1d878139df9e39e8638e3a7ad\": container with ID starting with 93857e902c71602b2687958f62ffb78e7a0672d1d878139df9e39e8638e3a7ad not found: ID does not exist"
Jul 28 20:24:36.864428 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.864407    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43d62f69-3682-473f-842d-bdbd6a2005b1" path="/var/lib/kubelet/pods/43d62f69-3682-473f-842d-bdbd6a2005b1/volumes"
Jul 28 20:24:36.873090 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.873034    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"]
Jul 28 20:24:36.882338 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:36.882312    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d170bd/sub-test-model-d170bd-kserve-8697486bc5-4tgpp"]
Jul 28 20:24:38.864315 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:38.864274    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="169c5629-e446-4612-b1fc-27b269508c4d" path="/var/lib/kubelet/pods/169c5629-e446-4612-b1fc-27b269508c4d/volumes"
Jul 28 20:24:50.012218 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.012187    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"]
Jul 28 20:24:50.012703 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.012684    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="169c5629-e446-4612-b1fc-27b269508c4d" containerName="main"
Jul 28 20:24:50.012783 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.012706    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="169c5629-e446-4612-b1fc-27b269508c4d" containerName="main"
Jul 28 20:24:50.012783 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.012743    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43d62f69-3682-473f-842d-bdbd6a2005b1" containerName="main"
Jul 28 20:24:50.012783 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.012754    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="43d62f69-3682-473f-842d-bdbd6a2005b1" containerName="main"
Jul 28 20:24:50.012933 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.012863    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="169c5629-e446-4612-b1fc-27b269508c4d" containerName="main"
Jul 28 20:24:50.012933 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.012877    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="43d62f69-3682-473f-842d-bdbd6a2005b1" containerName="main"
Jul 28 20:24:50.015817 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.015796    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.018449 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.018426    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d170bd\"/\"kube-root-ca.crt\""
Jul 28 20:24:50.019489 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.019471    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d170bd\"/\"rate-test-model-d170bd-kserve-self-signed-certs\""
Jul 28 20:24:50.019615 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.019521    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d170bd\"/\"default-dockercfg-zp99p\""
Jul 28 20:24:50.019615 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.019543    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d170bd\"/\"openshift-service-ca.crt\""
Jul 28 20:24:50.026248 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.026227    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"]
Jul 28 20:24:50.041853 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.041817    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:24:50.106342 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.106321    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s968h\" (UniqueName: \"kubernetes.io/projected/3b2bef7c-3096-4ebb-a1d0-17510295392c-kube-api-access-s968h\") pod \"rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") " pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.106452 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.106349    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-home\") pod \"rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") " pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.106452 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.106373    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-model-cache\") pod \"rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") " pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.106452 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.106388    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-tmp-dir\") pod \"rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") " pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.106662 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.106471    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-dshm\") pod \"rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") " pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.106662 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.106513    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b2bef7c-3096-4ebb-a1d0-17510295392c-tls-certs\") pod \"rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") " pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.207429 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.207404    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-dshm\") pod \"rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") " pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.207542 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.207438    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b2bef7c-3096-4ebb-a1d0-17510295392c-tls-certs\") pod \"rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") " pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.207542 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.207513    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s968h\" (UniqueName: \"kubernetes.io/projected/3b2bef7c-3096-4ebb-a1d0-17510295392c-kube-api-access-s968h\") pod \"rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") " pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.207542 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.207530    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-home\") pod \"rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") " pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.207716 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.207661    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-model-cache\") pod \"rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") " pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.207716 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.207687    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-tmp-dir\") pod \"rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") " pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.207921 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.207897    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-home\") pod \"rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") " pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.208027 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.208007    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-model-cache\") pod \"rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") " pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.208112 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.208016    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-tmp-dir\") pod \"rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") " pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.209767 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.209748    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-dshm\") pod \"rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") " pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.209941 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.209924    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b2bef7c-3096-4ebb-a1d0-17510295392c-tls-certs\") pod \"rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") " pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.216263 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.216244    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s968h\" (UniqueName: \"kubernetes.io/projected/3b2bef7c-3096-4ebb-a1d0-17510295392c-kube-api-access-s968h\") pod \"rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") " pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.325434 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.325407    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:24:50.479453 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.479422    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"]
Jul 28 20:24:50.483164 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:24:50.483136    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b2bef7c_3096_4ebb_a1d0_17510295392c.slice/crio-bbf75e4d737612978db5fe8716faa2a35eab6c45277ae48040fe21f2154a2473 WatchSource:0}: Error finding container bbf75e4d737612978db5fe8716faa2a35eab6c45277ae48040fe21f2154a2473: Status 404 returned error can't find the container with id bbf75e4d737612978db5fe8716faa2a35eab6c45277ae48040fe21f2154a2473
Jul 28 20:24:50.911625 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.911587    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2" event={"ID":"3b2bef7c-3096-4ebb-a1d0-17510295392c","Type":"ContainerStarted","Data":"294f881209712492b1f7644019a4718d3c635846d79e2ba4a758db361fc08e09"}
Jul 28 20:24:50.911625 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.911627    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2" event={"ID":"3b2bef7c-3096-4ebb-a1d0-17510295392c","Type":"ContainerStarted","Data":"bbf75e4d737612978db5fe8716faa2a35eab6c45277ae48040fe21f2154a2473"}
Jul 28 20:24:50.936017 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:24:50.935975    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2" podStartSLOduration=1.935959702 podStartE2EDuration="1.935959702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:24:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:24:50.934619653 +0000 UTC m=+3316.522282598" watchObservedRunningTime="2026-07-28 20:24:50.935959702 +0000 UTC m=+3316.523622660"
Jul 28 20:25:00.326250 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:25:00.326147    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:25:00.326250 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:25:00.326209    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:25:00.338674 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:25:00.338641    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:25:00.976296 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:25:00.976273    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:26:20.031238 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:20.031164    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:21.104535 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:21.104504    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"]
Jul 28 20:26:21.117874 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:21.117846    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:22.069788 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:22.069753    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7631-139926542746160"]
Jul 28 20:26:22.080888 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:22.080836    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542746160"
Jul 28 20:26:22.081612 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:22.081520    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7631-139926542746160"]
Jul 28 20:26:22.083702 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:22.083667    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8ljcb\""
Jul 28 20:26:22.140365 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:22.140340    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79m4l\" (UniqueName: \"kubernetes.io/projected/87994d0e-5623-4761-970f-c9eb7a220870-kube-api-access-79m4l\") pod \"test-curl-7631-139926542746160\" (UID: \"87994d0e-5623-4761-970f-c9eb7a220870\") " pod="opendatahub/test-curl-7631-139926542746160"
Jul 28 20:26:22.241681 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:22.241657    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79m4l\" (UniqueName: \"kubernetes.io/projected/87994d0e-5623-4761-970f-c9eb7a220870-kube-api-access-79m4l\") pod \"test-curl-7631-139926542746160\" (UID: \"87994d0e-5623-4761-970f-c9eb7a220870\") " pod="opendatahub/test-curl-7631-139926542746160"
Jul 28 20:26:22.250620 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:22.250597    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79m4l\" (UniqueName: \"kubernetes.io/projected/87994d0e-5623-4761-970f-c9eb7a220870-kube-api-access-79m4l\") pod \"test-curl-7631-139926542746160\" (UID: \"87994d0e-5623-4761-970f-c9eb7a220870\") " pod="opendatahub/test-curl-7631-139926542746160"
Jul 28 20:26:22.390784 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:22.390721    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542746160"
Jul 28 20:26:22.522456 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:22.522425    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7631-139926542746160"]
Jul 28 20:26:22.524275 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:26:22.524250    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87994d0e_5623_4761_970f_c9eb7a220870.slice/crio-b44bca64fb0b03277295eb45f8f20316c737cbbff9e2ffbc74295e594ba4af31 WatchSource:0}: Error finding container b44bca64fb0b03277295eb45f8f20316c737cbbff9e2ffbc74295e594ba4af31: Status 404 returned error can't find the container with id b44bca64fb0b03277295eb45f8f20316c737cbbff9e2ffbc74295e594ba4af31
Jul 28 20:26:22.525486 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:22.525469    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:26:23.421834 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:23.421805    2572 generic.go:356] "Generic (PLEG): container finished" podID="87994d0e-5623-4761-970f-c9eb7a220870" containerID="e0d18144f85afb3957c99e34ca67f73fe0d17c09e882407013281a30caff2371" exitCode=0
Jul 28 20:26:23.422162 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:23.421853    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7631-139926542746160" event={"ID":"87994d0e-5623-4761-970f-c9eb7a220870","Type":"ContainerDied","Data":"e0d18144f85afb3957c99e34ca67f73fe0d17c09e882407013281a30caff2371"}
Jul 28 20:26:23.422162 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:23.421887    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7631-139926542746160" event={"ID":"87994d0e-5623-4761-970f-c9eb7a220870","Type":"ContainerStarted","Data":"b44bca64fb0b03277295eb45f8f20316c737cbbff9e2ffbc74295e594ba4af31"}
Jul 28 20:26:24.561116 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:24.561090    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542746160"
Jul 28 20:26:24.581642 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:24.581621    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7631-139926542746160_87994d0e-5623-4761-970f-c9eb7a220870/test-curl-7631-139926542746160/0.log"
Jul 28 20:26:24.605028 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:24.605004    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7631-139926542746160"]
Jul 28 20:26:24.608754 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:24.608730    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7631-139926542746160"]
Jul 28 20:26:24.661870 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:24.661846    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79m4l\" (UniqueName: \"kubernetes.io/projected/87994d0e-5623-4761-970f-c9eb7a220870-kube-api-access-79m4l\") pod \"87994d0e-5623-4761-970f-c9eb7a220870\" (UID: \"87994d0e-5623-4761-970f-c9eb7a220870\") "
Jul 28 20:26:24.663838 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:24.663815    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87994d0e-5623-4761-970f-c9eb7a220870-kube-api-access-79m4l" (OuterVolumeSpecName: "kube-api-access-79m4l") pod "87994d0e-5623-4761-970f-c9eb7a220870" (UID: "87994d0e-5623-4761-970f-c9eb7a220870"). InnerVolumeSpecName "kube-api-access-79m4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:26:24.722714 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:24.722654    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7631-139926542647728"]
Jul 28 20:26:24.723022 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:24.723009    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87994d0e-5623-4761-970f-c9eb7a220870" containerName="test-curl-7631-139926542746160"
Jul 28 20:26:24.723063 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:24.723023    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="87994d0e-5623-4761-970f-c9eb7a220870" containerName="test-curl-7631-139926542746160"
Jul 28 20:26:24.723130 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:24.723099    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="87994d0e-5623-4761-970f-c9eb7a220870" containerName="test-curl-7631-139926542746160"
Jul 28 20:26:24.726133 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:24.726116    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542647728"
Jul 28 20:26:24.730094 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:24.730059    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7631-139926542647728"]
Jul 28 20:26:24.763184 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:24.763160    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blhgn\" (UniqueName: \"kubernetes.io/projected/c9b28bca-a486-4dec-8a88-9f8e7953e06b-kube-api-access-blhgn\") pod \"test-curl-7631-139926542647728\" (UID: \"c9b28bca-a486-4dec-8a88-9f8e7953e06b\") " pod="opendatahub/test-curl-7631-139926542647728"
Jul 28 20:26:24.763274 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:24.763252    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79m4l\" (UniqueName: \"kubernetes.io/projected/87994d0e-5623-4761-970f-c9eb7a220870-kube-api-access-79m4l\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:26:24.863461 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:24.863435    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87994d0e-5623-4761-970f-c9eb7a220870" path="/var/lib/kubelet/pods/87994d0e-5623-4761-970f-c9eb7a220870/volumes"
Jul 28 20:26:24.863701 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:24.863681    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blhgn\" (UniqueName: \"kubernetes.io/projected/c9b28bca-a486-4dec-8a88-9f8e7953e06b-kube-api-access-blhgn\") pod \"test-curl-7631-139926542647728\" (UID: \"c9b28bca-a486-4dec-8a88-9f8e7953e06b\") " pod="opendatahub/test-curl-7631-139926542647728"
Jul 28 20:26:24.872164 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:24.872145    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blhgn\" (UniqueName: \"kubernetes.io/projected/c9b28bca-a486-4dec-8a88-9f8e7953e06b-kube-api-access-blhgn\") pod \"test-curl-7631-139926542647728\" (UID: \"c9b28bca-a486-4dec-8a88-9f8e7953e06b\") " pod="opendatahub/test-curl-7631-139926542647728"
Jul 28 20:26:25.036521 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:25.036462    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542647728"
Jul 28 20:26:25.169270 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:26:25.169242    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9b28bca_a486_4dec_8a88_9f8e7953e06b.slice/crio-680491746cd5b5283102bcf7f0321a8e0fecb4b504d1b2b1d84e21d743b2b0fe WatchSource:0}: Error finding container 680491746cd5b5283102bcf7f0321a8e0fecb4b504d1b2b1d84e21d743b2b0fe: Status 404 returned error can't find the container with id 680491746cd5b5283102bcf7f0321a8e0fecb4b504d1b2b1d84e21d743b2b0fe
Jul 28 20:26:25.169934 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:25.169910    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7631-139926542647728"]
Jul 28 20:26:25.433104 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:25.433062    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542746160"
Jul 28 20:26:25.433210 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:25.433117    2572 scope.go:117] "RemoveContainer" containerID="e0d18144f85afb3957c99e34ca67f73fe0d17c09e882407013281a30caff2371"
Jul 28 20:26:25.434481 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:25.434452    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7631-139926542647728" event={"ID":"c9b28bca-a486-4dec-8a88-9f8e7953e06b","Type":"ContainerStarted","Data":"680491746cd5b5283102bcf7f0321a8e0fecb4b504d1b2b1d84e21d743b2b0fe"}
Jul 28 20:26:26.441556 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:26.441521    2572 generic.go:356] "Generic (PLEG): container finished" podID="c9b28bca-a486-4dec-8a88-9f8e7953e06b" containerID="f91517e0aba72aea6228e3b3f274ead5ae15a5504dd4d4f74c92fdf4e077b3e6" exitCode=0
Jul 28 20:26:26.442008 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:26.441615    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7631-139926542647728" event={"ID":"c9b28bca-a486-4dec-8a88-9f8e7953e06b","Type":"ContainerDied","Data":"f91517e0aba72aea6228e3b3f274ead5ae15a5504dd4d4f74c92fdf4e077b3e6"}
Jul 28 20:26:27.124652 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:27.124619    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:27.582620 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:27.582594    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542647728"
Jul 28 20:26:27.602299 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:27.602275    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7631-139926542647728_c9b28bca-a486-4dec-8a88-9f8e7953e06b/test-curl-7631-139926542647728/0.log"
Jul 28 20:26:27.624534 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:27.624506    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7631-139926542647728"]
Jul 28 20:26:27.628214 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:27.628194    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7631-139926542647728"]
Jul 28 20:26:27.689239 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:27.689205    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blhgn\" (UniqueName: \"kubernetes.io/projected/c9b28bca-a486-4dec-8a88-9f8e7953e06b-kube-api-access-blhgn\") pod \"c9b28bca-a486-4dec-8a88-9f8e7953e06b\" (UID: \"c9b28bca-a486-4dec-8a88-9f8e7953e06b\") "
Jul 28 20:26:27.691165 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:27.691143    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9b28bca-a486-4dec-8a88-9f8e7953e06b-kube-api-access-blhgn" (OuterVolumeSpecName: "kube-api-access-blhgn") pod "c9b28bca-a486-4dec-8a88-9f8e7953e06b" (UID: "c9b28bca-a486-4dec-8a88-9f8e7953e06b"). InnerVolumeSpecName "kube-api-access-blhgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:26:27.784340 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:27.784283    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7631-139926542853440"]
Jul 28 20:26:27.784647 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:27.784634    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9b28bca-a486-4dec-8a88-9f8e7953e06b" containerName="test-curl-7631-139926542647728"
Jul 28 20:26:27.784706 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:27.784648    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9b28bca-a486-4dec-8a88-9f8e7953e06b" containerName="test-curl-7631-139926542647728"
Jul 28 20:26:27.784740 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:27.784710    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9b28bca-a486-4dec-8a88-9f8e7953e06b" containerName="test-curl-7631-139926542647728"
Jul 28 20:26:27.787683 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:27.787669    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542853440"
Jul 28 20:26:27.789935 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:27.789918    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blhgn\" (UniqueName: \"kubernetes.io/projected/c9b28bca-a486-4dec-8a88-9f8e7953e06b-kube-api-access-blhgn\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:26:27.794420 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:27.794399    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7631-139926542853440"]
Jul 28 20:26:27.890962 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:27.890931    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jdqs\" (UniqueName: \"kubernetes.io/projected/2addbc83-b8bc-4354-9de4-729d93a9fcd9-kube-api-access-2jdqs\") pod \"test-curl-7631-139926542853440\" (UID: \"2addbc83-b8bc-4354-9de4-729d93a9fcd9\") " pod="opendatahub/test-curl-7631-139926542853440"
Jul 28 20:26:27.992287 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:27.992258    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jdqs\" (UniqueName: \"kubernetes.io/projected/2addbc83-b8bc-4354-9de4-729d93a9fcd9-kube-api-access-2jdqs\") pod \"test-curl-7631-139926542853440\" (UID: \"2addbc83-b8bc-4354-9de4-729d93a9fcd9\") " pod="opendatahub/test-curl-7631-139926542853440"
Jul 28 20:26:28.000321 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:28.000298    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jdqs\" (UniqueName: \"kubernetes.io/projected/2addbc83-b8bc-4354-9de4-729d93a9fcd9-kube-api-access-2jdqs\") pod \"test-curl-7631-139926542853440\" (UID: \"2addbc83-b8bc-4354-9de4-729d93a9fcd9\") " pod="opendatahub/test-curl-7631-139926542853440"
Jul 28 20:26:28.098168 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:28.098143    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542853440"
Jul 28 20:26:28.231318 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:28.231293    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7631-139926542853440"]
Jul 28 20:26:28.232816 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:26:28.232788    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2addbc83_b8bc_4354_9de4_729d93a9fcd9.slice/crio-05e7148a21ffb51f1e4b7a74ebbaa3fb74a48e434030741987b4abd9c0dd8a83 WatchSource:0}: Error finding container 05e7148a21ffb51f1e4b7a74ebbaa3fb74a48e434030741987b4abd9c0dd8a83: Status 404 returned error can't find the container with id 05e7148a21ffb51f1e4b7a74ebbaa3fb74a48e434030741987b4abd9c0dd8a83
Jul 28 20:26:28.454175 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:28.454107    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542647728"
Jul 28 20:26:28.454175 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:28.454143    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="680491746cd5b5283102bcf7f0321a8e0fecb4b504d1b2b1d84e21d743b2b0fe"
Jul 28 20:26:28.455516 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:28.455477    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7631-139926542853440" event={"ID":"2addbc83-b8bc-4354-9de4-729d93a9fcd9","Type":"ContainerStarted","Data":"05e7148a21ffb51f1e4b7a74ebbaa3fb74a48e434030741987b4abd9c0dd8a83"}
Jul 28 20:26:28.464992 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:26:28.464966    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7631-139926542647728\" is forbidden: User \"system:node:ip-10-0-138-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-37.ec2.internal' and this object" podUID="c9b28bca-a486-4dec-8a88-9f8e7953e06b" pod="opendatahub/test-curl-7631-139926542647728"
Jul 28 20:26:28.864991 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:28.864961    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9b28bca-a486-4dec-8a88-9f8e7953e06b" path="/var/lib/kubelet/pods/c9b28bca-a486-4dec-8a88-9f8e7953e06b/volumes"
Jul 28 20:26:29.462512 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:29.462476    2572 generic.go:356] "Generic (PLEG): container finished" podID="2addbc83-b8bc-4354-9de4-729d93a9fcd9" containerID="ba161d79b510eb3ed7633e7a6ffc62ab15b480532f0875b997a1a9d198d26726" exitCode=0
Jul 28 20:26:29.462696 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:29.462558    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7631-139926542853440" event={"ID":"2addbc83-b8bc-4354-9de4-729d93a9fcd9","Type":"ContainerDied","Data":"ba161d79b510eb3ed7633e7a6ffc62ab15b480532f0875b997a1a9d198d26726"}
Jul 28 20:26:30.601266 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:30.601245    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542853440"
Jul 28 20:26:30.622241 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:30.622219    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7631-139926542853440_2addbc83-b8bc-4354-9de4-729d93a9fcd9/test-curl-7631-139926542853440/0.log"
Jul 28 20:26:30.643713 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:30.643683    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7631-139926542853440"]
Jul 28 20:26:30.647651 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:30.647630    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7631-139926542853440"]
Jul 28 20:26:30.713878 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:30.713852    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jdqs\" (UniqueName: \"kubernetes.io/projected/2addbc83-b8bc-4354-9de4-729d93a9fcd9-kube-api-access-2jdqs\") pod \"2addbc83-b8bc-4354-9de4-729d93a9fcd9\" (UID: \"2addbc83-b8bc-4354-9de4-729d93a9fcd9\") "
Jul 28 20:26:30.715765 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:30.715744    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2addbc83-b8bc-4354-9de4-729d93a9fcd9-kube-api-access-2jdqs" (OuterVolumeSpecName: "kube-api-access-2jdqs") pod "2addbc83-b8bc-4354-9de4-729d93a9fcd9" (UID: "2addbc83-b8bc-4354-9de4-729d93a9fcd9"). InnerVolumeSpecName "kube-api-access-2jdqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:26:30.763703 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:30.763678    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7631-139926542853440"]
Jul 28 20:26:30.764065 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:30.764052    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2addbc83-b8bc-4354-9de4-729d93a9fcd9" containerName="test-curl-7631-139926542853440"
Jul 28 20:26:30.764065 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:30.764079    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2addbc83-b8bc-4354-9de4-729d93a9fcd9" containerName="test-curl-7631-139926542853440"
Jul 28 20:26:30.764166 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:30.764147    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2addbc83-b8bc-4354-9de4-729d93a9fcd9" containerName="test-curl-7631-139926542853440"
Jul 28 20:26:30.767009 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:30.766995    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542853440"
Jul 28 20:26:30.773453 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:30.773434    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7631-139926542853440"]
Jul 28 20:26:30.814670 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:30.814647    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27mbq\" (UniqueName: \"kubernetes.io/projected/f453f2f0-c01b-4bba-961f-dde25fd17bd1-kube-api-access-27mbq\") pod \"test-curl-7631-139926542853440\" (UID: \"f453f2f0-c01b-4bba-961f-dde25fd17bd1\") " pod="opendatahub/test-curl-7631-139926542853440"
Jul 28 20:26:30.814780 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:30.814706    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jdqs\" (UniqueName: \"kubernetes.io/projected/2addbc83-b8bc-4354-9de4-729d93a9fcd9-kube-api-access-2jdqs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:26:30.864737 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:30.864708    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2addbc83-b8bc-4354-9de4-729d93a9fcd9" path="/var/lib/kubelet/pods/2addbc83-b8bc-4354-9de4-729d93a9fcd9/volumes"
Jul 28 20:26:30.915847 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:30.915819    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27mbq\" (UniqueName: \"kubernetes.io/projected/f453f2f0-c01b-4bba-961f-dde25fd17bd1-kube-api-access-27mbq\") pod \"test-curl-7631-139926542853440\" (UID: \"f453f2f0-c01b-4bba-961f-dde25fd17bd1\") " pod="opendatahub/test-curl-7631-139926542853440"
Jul 28 20:26:30.925204 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:30.925179    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27mbq\" (UniqueName: \"kubernetes.io/projected/f453f2f0-c01b-4bba-961f-dde25fd17bd1-kube-api-access-27mbq\") pod \"test-curl-7631-139926542853440\" (UID: \"f453f2f0-c01b-4bba-961f-dde25fd17bd1\") " pod="opendatahub/test-curl-7631-139926542853440"
Jul 28 20:26:31.077030 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:31.077008    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542853440"
Jul 28 20:26:31.206270 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:31.206243    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7631-139926542853440"]
Jul 28 20:26:31.207833 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:26:31.207806    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf453f2f0_c01b_4bba_961f_dde25fd17bd1.slice/crio-e5a011fdab036bd15194a1977cd8804900d986fc508212e0a578b1b799db0e4a WatchSource:0}: Error finding container e5a011fdab036bd15194a1977cd8804900d986fc508212e0a578b1b799db0e4a: Status 404 returned error can't find the container with id e5a011fdab036bd15194a1977cd8804900d986fc508212e0a578b1b799db0e4a
Jul 28 20:26:31.475434 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:31.475353    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7631-139926542853440" event={"ID":"f453f2f0-c01b-4bba-961f-dde25fd17bd1","Type":"ContainerStarted","Data":"e5a011fdab036bd15194a1977cd8804900d986fc508212e0a578b1b799db0e4a"}
Jul 28 20:26:31.476697 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:31.476677    2572 scope.go:117] "RemoveContainer" containerID="ba161d79b510eb3ed7633e7a6ffc62ab15b480532f0875b997a1a9d198d26726"
Jul 28 20:26:31.476803 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:31.476703    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542853440"
Jul 28 20:26:32.485932 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:32.485890    2572 generic.go:356] "Generic (PLEG): container finished" podID="f453f2f0-c01b-4bba-961f-dde25fd17bd1" containerID="e44bcb9b9064760f5611573353133a186cf0073f75a816de8ff92765613cb892" exitCode=0
Jul 28 20:26:32.486400 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:32.485982    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7631-139926542853440" event={"ID":"f453f2f0-c01b-4bba-961f-dde25fd17bd1","Type":"ContainerDied","Data":"e44bcb9b9064760f5611573353133a186cf0073f75a816de8ff92765613cb892"}
Jul 28 20:26:33.627339 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:33.627319    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542853440"
Jul 28 20:26:33.648112 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:33.648083    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7631-139926542853440_f453f2f0-c01b-4bba-961f-dde25fd17bd1/test-curl-7631-139926542853440/0.log"
Jul 28 20:26:33.674444 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:33.674420    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7631-139926542853440"]
Jul 28 20:26:33.676605 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:33.676586    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7631-139926542853440"]
Jul 28 20:26:33.739796 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:33.739775    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27mbq\" (UniqueName: \"kubernetes.io/projected/f453f2f0-c01b-4bba-961f-dde25fd17bd1-kube-api-access-27mbq\") pod \"f453f2f0-c01b-4bba-961f-dde25fd17bd1\" (UID: \"f453f2f0-c01b-4bba-961f-dde25fd17bd1\") "
Jul 28 20:26:33.741716 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:33.741695    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f453f2f0-c01b-4bba-961f-dde25fd17bd1-kube-api-access-27mbq" (OuterVolumeSpecName: "kube-api-access-27mbq") pod "f453f2f0-c01b-4bba-961f-dde25fd17bd1" (UID: "f453f2f0-c01b-4bba-961f-dde25fd17bd1"). InnerVolumeSpecName "kube-api-access-27mbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:26:33.792060 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:33.792037    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7631-139926542855312"]
Jul 28 20:26:33.792402 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:33.792389    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f453f2f0-c01b-4bba-961f-dde25fd17bd1" containerName="test-curl-7631-139926542853440"
Jul 28 20:26:33.792439 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:33.792404    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f453f2f0-c01b-4bba-961f-dde25fd17bd1" containerName="test-curl-7631-139926542853440"
Jul 28 20:26:33.792485 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:33.792476    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f453f2f0-c01b-4bba-961f-dde25fd17bd1" containerName="test-curl-7631-139926542853440"
Jul 28 20:26:33.795427 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:33.795413    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542855312"
Jul 28 20:26:33.800235 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:33.800209    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7631-139926542855312"]
Jul 28 20:26:33.840357 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:33.840336    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hg6k\" (UniqueName: \"kubernetes.io/projected/fb6672ba-d359-4982-b739-445226270d94-kube-api-access-4hg6k\") pod \"test-curl-7631-139926542855312\" (UID: \"fb6672ba-d359-4982-b739-445226270d94\") " pod="opendatahub/test-curl-7631-139926542855312"
Jul 28 20:26:33.840454 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:33.840376    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27mbq\" (UniqueName: \"kubernetes.io/projected/f453f2f0-c01b-4bba-961f-dde25fd17bd1-kube-api-access-27mbq\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:26:33.940923 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:33.940897    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hg6k\" (UniqueName: \"kubernetes.io/projected/fb6672ba-d359-4982-b739-445226270d94-kube-api-access-4hg6k\") pod \"test-curl-7631-139926542855312\" (UID: \"fb6672ba-d359-4982-b739-445226270d94\") " pod="opendatahub/test-curl-7631-139926542855312"
Jul 28 20:26:33.949291 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:33.949269    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hg6k\" (UniqueName: \"kubernetes.io/projected/fb6672ba-d359-4982-b739-445226270d94-kube-api-access-4hg6k\") pod \"test-curl-7631-139926542855312\" (UID: \"fb6672ba-d359-4982-b739-445226270d94\") " pod="opendatahub/test-curl-7631-139926542855312"
Jul 28 20:26:34.105552 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:34.105530    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542855312"
Jul 28 20:26:34.234501 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:34.234478    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7631-139926542855312"]
Jul 28 20:26:34.235725 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:26:34.235700    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb6672ba_d359_4982_b739_445226270d94.slice/crio-f488c1c03c855eac59e4d0003bd8237bb97a82c6c03d4781e8461587213c89d7 WatchSource:0}: Error finding container f488c1c03c855eac59e4d0003bd8237bb97a82c6c03d4781e8461587213c89d7: Status 404 returned error can't find the container with id f488c1c03c855eac59e4d0003bd8237bb97a82c6c03d4781e8461587213c89d7
Jul 28 20:26:34.499895 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:34.499833    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7631-139926542855312" event={"ID":"fb6672ba-d359-4982-b739-445226270d94","Type":"ContainerStarted","Data":"f488c1c03c855eac59e4d0003bd8237bb97a82c6c03d4781e8461587213c89d7"}
Jul 28 20:26:34.501123 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:34.501100    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e5a011fdab036bd15194a1977cd8804900d986fc508212e0a578b1b799db0e4a"
Jul 28 20:26:34.501123 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:34.501116    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542853440"
Jul 28 20:26:34.511945 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:26:34.511916    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7631-139926542853440\" is forbidden: User \"system:node:ip-10-0-138-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-37.ec2.internal' and this object" podUID="f453f2f0-c01b-4bba-961f-dde25fd17bd1" pod="opendatahub/test-curl-7631-139926542853440"
Jul 28 20:26:34.862537 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:26:34.862509    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7631-139926542853440\" is forbidden: User \"system:node:ip-10-0-138-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-37.ec2.internal' and this object" podUID="f453f2f0-c01b-4bba-961f-dde25fd17bd1" pod="opendatahub/test-curl-7631-139926542853440"
Jul 28 20:26:34.865737 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:34.865714    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f453f2f0-c01b-4bba-961f-dde25fd17bd1" path="/var/lib/kubelet/pods/f453f2f0-c01b-4bba-961f-dde25fd17bd1/volumes"
Jul 28 20:26:35.507988 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:35.507915    2572 generic.go:356] "Generic (PLEG): container finished" podID="fb6672ba-d359-4982-b739-445226270d94" containerID="3dd3ffe3d8b91d4154861d62bf433755da0823657aa5c5db3fd852a8b307efe9" exitCode=0
Jul 28 20:26:35.508129 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:35.508017    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7631-139926542855312" event={"ID":"fb6672ba-d359-4982-b739-445226270d94","Type":"ContainerDied","Data":"3dd3ffe3d8b91d4154861d62bf433755da0823657aa5c5db3fd852a8b307efe9"}
Jul 28 20:26:36.123272 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.123239    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2" podUID="3b2bef7c-3096-4ebb-a1d0-17510295392c" containerName="main" containerID="cri-o://294f881209712492b1f7644019a4718d3c635846d79e2ba4a758db361fc08e09" gracePeriod=45
Jul 28 20:26:36.382803 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.382751    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:26:36.464768 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.464740    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-tmp-dir\") pod \"3b2bef7c-3096-4ebb-a1d0-17510295392c\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") "
Jul 28 20:26:36.464913 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.464774    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-home\") pod \"3b2bef7c-3096-4ebb-a1d0-17510295392c\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") "
Jul 28 20:26:36.464913 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.464794    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b2bef7c-3096-4ebb-a1d0-17510295392c-tls-certs\") pod \"3b2bef7c-3096-4ebb-a1d0-17510295392c\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") "
Jul 28 20:26:36.464913 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.464833    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-model-cache\") pod \"3b2bef7c-3096-4ebb-a1d0-17510295392c\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") "
Jul 28 20:26:36.464913 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.464849    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s968h\" (UniqueName: \"kubernetes.io/projected/3b2bef7c-3096-4ebb-a1d0-17510295392c-kube-api-access-s968h\") pod \"3b2bef7c-3096-4ebb-a1d0-17510295392c\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") "
Jul 28 20:26:36.465152 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.464955    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-dshm\") pod \"3b2bef7c-3096-4ebb-a1d0-17510295392c\" (UID: \"3b2bef7c-3096-4ebb-a1d0-17510295392c\") "
Jul 28 20:26:36.465152 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.465025    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-home" (OuterVolumeSpecName: "home") pod "3b2bef7c-3096-4ebb-a1d0-17510295392c" (UID: "3b2bef7c-3096-4ebb-a1d0-17510295392c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:26:36.465152 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.465041    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3b2bef7c-3096-4ebb-a1d0-17510295392c" (UID: "3b2bef7c-3096-4ebb-a1d0-17510295392c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:26:36.465261 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.465149    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-model-cache" (OuterVolumeSpecName: "model-cache") pod "3b2bef7c-3096-4ebb-a1d0-17510295392c" (UID: "3b2bef7c-3096-4ebb-a1d0-17510295392c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:26:36.465341 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.465321    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-tmp-dir\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:26:36.465341 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.465340    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-home\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:26:36.465448 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.465348    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-model-cache\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:26:36.466831 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.466811    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-dshm" (OuterVolumeSpecName: "dshm") pod "3b2bef7c-3096-4ebb-a1d0-17510295392c" (UID: "3b2bef7c-3096-4ebb-a1d0-17510295392c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:26:36.466932 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.466910    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b2bef7c-3096-4ebb-a1d0-17510295392c-kube-api-access-s968h" (OuterVolumeSpecName: "kube-api-access-s968h") pod "3b2bef7c-3096-4ebb-a1d0-17510295392c" (UID: "3b2bef7c-3096-4ebb-a1d0-17510295392c"). InnerVolumeSpecName "kube-api-access-s968h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:26:36.467001 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.466980    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b2bef7c-3096-4ebb-a1d0-17510295392c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3b2bef7c-3096-4ebb-a1d0-17510295392c" (UID: "3b2bef7c-3096-4ebb-a1d0-17510295392c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:26:36.513746 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.513722    2572 generic.go:356] "Generic (PLEG): container finished" podID="3b2bef7c-3096-4ebb-a1d0-17510295392c" containerID="294f881209712492b1f7644019a4718d3c635846d79e2ba4a758db361fc08e09" exitCode=0
Jul 28 20:26:36.513835 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.513764    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"
Jul 28 20:26:36.513835 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.513817    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2" event={"ID":"3b2bef7c-3096-4ebb-a1d0-17510295392c","Type":"ContainerDied","Data":"294f881209712492b1f7644019a4718d3c635846d79e2ba4a758db361fc08e09"}
Jul 28 20:26:36.514005 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.513863    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2" event={"ID":"3b2bef7c-3096-4ebb-a1d0-17510295392c","Type":"ContainerDied","Data":"bbf75e4d737612978db5fe8716faa2a35eab6c45277ae48040fe21f2154a2473"}
Jul 28 20:26:36.514005 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.513890    2572 scope.go:117] "RemoveContainer" containerID="294f881209712492b1f7644019a4718d3c635846d79e2ba4a758db361fc08e09"
Jul 28 20:26:36.523805 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.523788    2572 scope.go:117] "RemoveContainer" containerID="294f881209712492b1f7644019a4718d3c635846d79e2ba4a758db361fc08e09"
Jul 28 20:26:36.524109 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:26:36.524055    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"294f881209712492b1f7644019a4718d3c635846d79e2ba4a758db361fc08e09\": container with ID starting with 294f881209712492b1f7644019a4718d3c635846d79e2ba4a758db361fc08e09 not found: ID does not exist" containerID="294f881209712492b1f7644019a4718d3c635846d79e2ba4a758db361fc08e09"
Jul 28 20:26:36.524203 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.524119    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"294f881209712492b1f7644019a4718d3c635846d79e2ba4a758db361fc08e09"} err="failed to get container status \"294f881209712492b1f7644019a4718d3c635846d79e2ba4a758db361fc08e09\": rpc error: code = NotFound desc = could not find container \"294f881209712492b1f7644019a4718d3c635846d79e2ba4a758db361fc08e09\": container with ID starting with 294f881209712492b1f7644019a4718d3c635846d79e2ba4a758db361fc08e09 not found: ID does not exist"
Jul 28 20:26:36.540036 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.540012    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"]
Jul 28 20:26:36.543800 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.543779    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d170bd/rate-test-model-d170bd-kserve-5dfddd56f9-8cmk2"]
Jul 28 20:26:36.566453 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.566433    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s968h\" (UniqueName: \"kubernetes.io/projected/3b2bef7c-3096-4ebb-a1d0-17510295392c-kube-api-access-s968h\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:26:36.566453 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.566454    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b2bef7c-3096-4ebb-a1d0-17510295392c-dshm\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:26:36.566580 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.566465    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b2bef7c-3096-4ebb-a1d0-17510295392c-tls-certs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:26:36.636491 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.636433    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542855312"
Jul 28 20:26:36.655944 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.655921    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7631-139926542855312_fb6672ba-d359-4982-b739-445226270d94/test-curl-7631-139926542855312/0.log"
Jul 28 20:26:36.678710 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.678684    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7631-139926542855312"]
Jul 28 20:26:36.682879 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.682852    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7631-139926542855312"]
Jul 28 20:26:36.767676 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.767651    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hg6k\" (UniqueName: \"kubernetes.io/projected/fb6672ba-d359-4982-b739-445226270d94-kube-api-access-4hg6k\") pod \"fb6672ba-d359-4982-b739-445226270d94\" (UID: \"fb6672ba-d359-4982-b739-445226270d94\") "
Jul 28 20:26:36.769664 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.769643    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb6672ba-d359-4982-b739-445226270d94-kube-api-access-4hg6k" (OuterVolumeSpecName: "kube-api-access-4hg6k") pod "fb6672ba-d359-4982-b739-445226270d94" (UID: "fb6672ba-d359-4982-b739-445226270d94"). InnerVolumeSpecName "kube-api-access-4hg6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:26:36.795927 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.795904    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7631-139926542855312"]
Jul 28 20:26:36.796309 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.796296    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b2bef7c-3096-4ebb-a1d0-17510295392c" containerName="main"
Jul 28 20:26:36.796356 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.796311    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b2bef7c-3096-4ebb-a1d0-17510295392c" containerName="main"
Jul 28 20:26:36.796356 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.796321    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb6672ba-d359-4982-b739-445226270d94" containerName="test-curl-7631-139926542855312"
Jul 28 20:26:36.796356 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.796327    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb6672ba-d359-4982-b739-445226270d94" containerName="test-curl-7631-139926542855312"
Jul 28 20:26:36.796453 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.796394    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b2bef7c-3096-4ebb-a1d0-17510295392c" containerName="main"
Jul 28 20:26:36.796453 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.796405    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb6672ba-d359-4982-b739-445226270d94" containerName="test-curl-7631-139926542855312"
Jul 28 20:26:36.799393 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.799378    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542855312"
Jul 28 20:26:36.804466 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.804445    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7631-139926542855312"]
Jul 28 20:26:36.864895 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.864874    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b2bef7c-3096-4ebb-a1d0-17510295392c" path="/var/lib/kubelet/pods/3b2bef7c-3096-4ebb-a1d0-17510295392c/volumes"
Jul 28 20:26:36.865217 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.865204    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb6672ba-d359-4982-b739-445226270d94" path="/var/lib/kubelet/pods/fb6672ba-d359-4982-b739-445226270d94/volumes"
Jul 28 20:26:36.868232 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.868208    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpjkh\" (UniqueName: \"kubernetes.io/projected/1d89af7d-aa98-4c0c-8ce7-b0ed26301d84-kube-api-access-xpjkh\") pod \"test-curl-7631-139926542855312\" (UID: \"1d89af7d-aa98-4c0c-8ce7-b0ed26301d84\") " pod="opendatahub/test-curl-7631-139926542855312"
Jul 28 20:26:36.868344 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.868329    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hg6k\" (UniqueName: \"kubernetes.io/projected/fb6672ba-d359-4982-b739-445226270d94-kube-api-access-4hg6k\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:26:36.969107 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.969026    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpjkh\" (UniqueName: \"kubernetes.io/projected/1d89af7d-aa98-4c0c-8ce7-b0ed26301d84-kube-api-access-xpjkh\") pod \"test-curl-7631-139926542855312\" (UID: \"1d89af7d-aa98-4c0c-8ce7-b0ed26301d84\") " pod="opendatahub/test-curl-7631-139926542855312"
Jul 28 20:26:36.977781 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:36.977763    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpjkh\" (UniqueName: \"kubernetes.io/projected/1d89af7d-aa98-4c0c-8ce7-b0ed26301d84-kube-api-access-xpjkh\") pod \"test-curl-7631-139926542855312\" (UID: \"1d89af7d-aa98-4c0c-8ce7-b0ed26301d84\") " pod="opendatahub/test-curl-7631-139926542855312"
Jul 28 20:26:37.109688 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:37.109669    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542855312"
Jul 28 20:26:37.236393 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:37.236362    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7631-139926542855312"]
Jul 28 20:26:37.237970 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:26:37.237945    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d89af7d_aa98_4c0c_8ce7_b0ed26301d84.slice/crio-199fb33ff7dbf97d893f9d6cdf760f08378513f21a993ec0286ed8a190d2ab70 WatchSource:0}: Error finding container 199fb33ff7dbf97d893f9d6cdf760f08378513f21a993ec0286ed8a190d2ab70: Status 404 returned error can't find the container with id 199fb33ff7dbf97d893f9d6cdf760f08378513f21a993ec0286ed8a190d2ab70
Jul 28 20:26:37.523804 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:37.523743    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7631-139926542855312" event={"ID":"1d89af7d-aa98-4c0c-8ce7-b0ed26301d84","Type":"ContainerStarted","Data":"199fb33ff7dbf97d893f9d6cdf760f08378513f21a993ec0286ed8a190d2ab70"}
Jul 28 20:26:37.524945 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:37.524929    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542855312"
Jul 28 20:26:37.525024 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:37.524954    2572 scope.go:117] "RemoveContainer" containerID="3dd3ffe3d8b91d4154861d62bf433755da0823657aa5c5db3fd852a8b307efe9"
Jul 28 20:26:38.533247 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:38.533215    2572 generic.go:356] "Generic (PLEG): container finished" podID="1d89af7d-aa98-4c0c-8ce7-b0ed26301d84" containerID="3fa709994fa8535ebd8a6088cbc17dcc16e6b7c3f0056d6ef46434bb9df3f5aa" exitCode=0
Jul 28 20:26:38.533662 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:38.533301    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7631-139926542855312" event={"ID":"1d89af7d-aa98-4c0c-8ce7-b0ed26301d84","Type":"ContainerDied","Data":"3fa709994fa8535ebd8a6088cbc17dcc16e6b7c3f0056d6ef46434bb9df3f5aa"}
Jul 28 20:26:39.699227 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:39.699199    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542855312"
Jul 28 20:26:39.722213 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:39.722188    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7631-139926542855312_1d89af7d-aa98-4c0c-8ce7-b0ed26301d84/test-curl-7631-139926542855312/0.log"
Jul 28 20:26:39.746242 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:39.746217    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7631-139926542855312"]
Jul 28 20:26:39.750223 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:39.750193    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7631-139926542855312"]
Jul 28 20:26:39.789291 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:39.789246    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpjkh\" (UniqueName: \"kubernetes.io/projected/1d89af7d-aa98-4c0c-8ce7-b0ed26301d84-kube-api-access-xpjkh\") pod \"1d89af7d-aa98-4c0c-8ce7-b0ed26301d84\" (UID: \"1d89af7d-aa98-4c0c-8ce7-b0ed26301d84\") "
Jul 28 20:26:39.791237 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:39.791217    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d89af7d-aa98-4c0c-8ce7-b0ed26301d84-kube-api-access-xpjkh" (OuterVolumeSpecName: "kube-api-access-xpjkh") pod "1d89af7d-aa98-4c0c-8ce7-b0ed26301d84" (UID: "1d89af7d-aa98-4c0c-8ce7-b0ed26301d84"). InnerVolumeSpecName "kube-api-access-xpjkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:26:39.890777 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:39.890752    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpjkh\" (UniqueName: \"kubernetes.io/projected/1d89af7d-aa98-4c0c-8ce7-b0ed26301d84-kube-api-access-xpjkh\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:26:40.250390 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:40.250358    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:40.550008 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:40.549941    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7631-139926542855312"
Jul 28 20:26:40.550008 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:40.549978    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="199fb33ff7dbf97d893f9d6cdf760f08378513f21a993ec0286ed8a190d2ab70"
Jul 28 20:26:40.564161 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:26:40.564138    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7631-139926542855312\" is forbidden: User \"system:node:ip-10-0-138-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-37.ec2.internal' and this object" podUID="1d89af7d-aa98-4c0c-8ce7-b0ed26301d84" pod="opendatahub/test-curl-7631-139926542855312"
Jul 28 20:26:40.615132 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:40.615108    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:40.865778 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:40.865747    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d89af7d-aa98-4c0c-8ce7-b0ed26301d84" path="/var/lib/kubelet/pods/1d89af7d-aa98-4c0c-8ce7-b0ed26301d84/volumes"
Jul 28 20:26:41.639303 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:41.639269    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:44.856809 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:44.856773    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:47.158272 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:47.158234    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:47.490512 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:47.490430    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:47.590209 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:47.590174    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:47.922275 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:47.922240    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:48.027876 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:26:48.027847    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:11.969796 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:11.969758    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:12.635956 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:12.635917    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:13.024559 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:13.024474    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:13.399990 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:13.399950    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:16.176625 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.176592    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:16.262419 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.262388    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"]
Jul 28 20:27:16.262776 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.262762    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d89af7d-aa98-4c0c-8ce7-b0ed26301d84" containerName="test-curl-7631-139926542855312"
Jul 28 20:27:16.262841 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.262777    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d89af7d-aa98-4c0c-8ce7-b0ed26301d84" containerName="test-curl-7631-139926542855312"
Jul 28 20:27:16.262896 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.262846    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d89af7d-aa98-4c0c-8ce7-b0ed26301d84" containerName="test-curl-7631-139926542855312"
Jul 28 20:27:16.264312 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.264285    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:16.267795 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.267778    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"
Jul 28 20:27:16.270880 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.270861    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-426065-serving-cert\""
Jul 28 20:27:16.277378 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.277356    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"]
Jul 28 20:27:16.331884 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.331863    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:16.390981 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.390951    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7ccdb8a-4a89-439a-91a0-615677e49ee3-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v\" (UID: \"f7ccdb8a-4a89-439a-91a0-615677e49ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"
Jul 28 20:27:16.391127 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.391010    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7ccdb8a-4a89-439a-91a0-615677e49ee3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v\" (UID: \"f7ccdb8a-4a89-439a-91a0-615677e49ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"
Jul 28 20:27:16.391207 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.391171    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2gkd\" (UniqueName: \"kubernetes.io/projected/f7ccdb8a-4a89-439a-91a0-615677e49ee3-kube-api-access-p2gkd\") pod \"maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v\" (UID: \"f7ccdb8a-4a89-439a-91a0-615677e49ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"
Jul 28 20:27:16.491732 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.491674    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7ccdb8a-4a89-439a-91a0-615677e49ee3-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v\" (UID: \"f7ccdb8a-4a89-439a-91a0-615677e49ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"
Jul 28 20:27:16.491732 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.491708    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7ccdb8a-4a89-439a-91a0-615677e49ee3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v\" (UID: \"f7ccdb8a-4a89-439a-91a0-615677e49ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"
Jul 28 20:27:16.491909 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.491740    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2gkd\" (UniqueName: \"kubernetes.io/projected/f7ccdb8a-4a89-439a-91a0-615677e49ee3-kube-api-access-p2gkd\") pod \"maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v\" (UID: \"f7ccdb8a-4a89-439a-91a0-615677e49ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"
Jul 28 20:27:16.491909 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:27:16.491851    2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-serving-cert: secret "maas-api-e2e-ipp-b-426065-serving-cert" not found
Jul 28 20:27:16.492016 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:27:16.491920    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7ccdb8a-4a89-439a-91a0-615677e49ee3-maas-api-tls podName:f7ccdb8a-4a89-439a-91a0-615677e49ee3 nodeName:}" failed. No retries permitted until 2026-07-28 20:27:16.991899521 +0000 UTC m=+3462.579562450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f7ccdb8a-4a89-439a-91a0-615677e49ee3-maas-api-tls") pod "maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v" (UID: "f7ccdb8a-4a89-439a-91a0-615677e49ee3") : secret "maas-api-e2e-ipp-b-426065-serving-cert" not found
Jul 28 20:27:16.492372 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.492353    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7ccdb8a-4a89-439a-91a0-615677e49ee3-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v\" (UID: \"f7ccdb8a-4a89-439a-91a0-615677e49ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"
Jul 28 20:27:16.501213 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.501191    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2gkd\" (UniqueName: \"kubernetes.io/projected/f7ccdb8a-4a89-439a-91a0-615677e49ee3-kube-api-access-p2gkd\") pod \"maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v\" (UID: \"f7ccdb8a-4a89-439a-91a0-615677e49ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"
Jul 28 20:27:16.924656 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.924620    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:16.997246 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.997216    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7ccdb8a-4a89-439a-91a0-615677e49ee3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v\" (UID: \"f7ccdb8a-4a89-439a-91a0-615677e49ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"
Jul 28 20:27:16.999579 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:16.999551    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7ccdb8a-4a89-439a-91a0-615677e49ee3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v\" (UID: \"f7ccdb8a-4a89-439a-91a0-615677e49ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"
Jul 28 20:27:17.054919 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.054892    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp"]
Jul 28 20:27:17.058713 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.058699    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp"
Jul 28 20:27:17.062330 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.062310    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-426065-dockercfg-8x4c2\""
Jul 28 20:27:17.062441 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.062424    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-426065\""
Jul 28 20:27:17.084937 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.084909    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:17.085201 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.085179    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp"]
Jul 28 20:27:17.127498 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.127474    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b"]
Jul 28 20:27:17.130920 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.130906    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b"
Jul 28 20:27:17.150835 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.150806    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b"]
Jul 28 20:27:17.180715 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.180642    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"
Jul 28 20:27:17.198866 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.198780    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26276\" (UniqueName: \"kubernetes.io/projected/f010ebe6-cfad-4f18-8bcf-9f68b3e924ff-kube-api-access-26276\") pod \"payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp\" (UID: \"f010ebe6-cfad-4f18-8bcf-9f68b3e924ff\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp"
Jul 28 20:27:17.198866 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.198836    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f010ebe6-cfad-4f18-8bcf-9f68b3e924ff-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp\" (UID: \"f010ebe6-cfad-4f18-8bcf-9f68b3e924ff\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp"
Jul 28 20:27:17.299815 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.299783    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4132d533-d2c6-4aeb-a76d-2467f201317a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b\" (UID: \"4132d533-d2c6-4aeb-a76d-2467f201317a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b"
Jul 28 20:27:17.299962 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.299853    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbfk8\" (UniqueName: \"kubernetes.io/projected/4132d533-d2c6-4aeb-a76d-2467f201317a-kube-api-access-bbfk8\") pod \"payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b\" (UID: \"4132d533-d2c6-4aeb-a76d-2467f201317a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b"
Jul 28 20:27:17.300035 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.299971    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26276\" (UniqueName: \"kubernetes.io/projected/f010ebe6-cfad-4f18-8bcf-9f68b3e924ff-kube-api-access-26276\") pod \"payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp\" (UID: \"f010ebe6-cfad-4f18-8bcf-9f68b3e924ff\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp"
Jul 28 20:27:17.300115 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.300038    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f010ebe6-cfad-4f18-8bcf-9f68b3e924ff-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp\" (UID: \"f010ebe6-cfad-4f18-8bcf-9f68b3e924ff\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp"
Jul 28 20:27:17.300778 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.300761    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f010ebe6-cfad-4f18-8bcf-9f68b3e924ff-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp\" (UID: \"f010ebe6-cfad-4f18-8bcf-9f68b3e924ff\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp"
Jul 28 20:27:17.314524 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.314504    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26276\" (UniqueName: \"kubernetes.io/projected/f010ebe6-cfad-4f18-8bcf-9f68b3e924ff-kube-api-access-26276\") pod \"payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp\" (UID: \"f010ebe6-cfad-4f18-8bcf-9f68b3e924ff\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp"
Jul 28 20:27:17.333779 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.333755    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"]
Jul 28 20:27:17.335182 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:27:17.335158    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7ccdb8a_4a89_439a_91a0_615677e49ee3.slice/crio-42afdc014923a32879d88fc91d098f82f266ada8c1101d1c7de98378caa33911 WatchSource:0}: Error finding container 42afdc014923a32879d88fc91d098f82f266ada8c1101d1c7de98378caa33911: Status 404 returned error can't find the container with id 42afdc014923a32879d88fc91d098f82f266ada8c1101d1c7de98378caa33911
Jul 28 20:27:17.368650 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.368631    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp"
Jul 28 20:27:17.401217 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.401191    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4132d533-d2c6-4aeb-a76d-2467f201317a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b\" (UID: \"4132d533-d2c6-4aeb-a76d-2467f201317a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b"
Jul 28 20:27:17.401322 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.401229    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbfk8\" (UniqueName: \"kubernetes.io/projected/4132d533-d2c6-4aeb-a76d-2467f201317a-kube-api-access-bbfk8\") pod \"payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b\" (UID: \"4132d533-d2c6-4aeb-a76d-2467f201317a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b"
Jul 28 20:27:17.401819 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.401802    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4132d533-d2c6-4aeb-a76d-2467f201317a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b\" (UID: \"4132d533-d2c6-4aeb-a76d-2467f201317a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b"
Jul 28 20:27:17.426768 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.426740    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbfk8\" (UniqueName: \"kubernetes.io/projected/4132d533-d2c6-4aeb-a76d-2467f201317a-kube-api-access-bbfk8\") pod \"payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b\" (UID: \"4132d533-d2c6-4aeb-a76d-2467f201317a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b"
Jul 28 20:27:17.440580 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.440521    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b"
Jul 28 20:27:17.561171 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.561146    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp"]
Jul 28 20:27:17.563120 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:27:17.563088    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf010ebe6_cfad_4f18_8bcf_9f68b3e924ff.slice/crio-292d1909067454f9cbd1c6fb5babf0c4272a91382a0faf9925d7cbecac511bba WatchSource:0}: Error finding container 292d1909067454f9cbd1c6fb5babf0c4272a91382a0faf9925d7cbecac511bba: Status 404 returned error can't find the container with id 292d1909067454f9cbd1c6fb5babf0c4272a91382a0faf9925d7cbecac511bba
Jul 28 20:27:17.620444 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.620407    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b"]
Jul 28 20:27:17.622776 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:27:17.622744    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4132d533_d2c6_4aeb_a76d_2467f201317a.slice/crio-67b1b17bc45171557ebaf155f472af7e3516ba20ded8ca1ece3c28ecc5615fc4 WatchSource:0}: Error finding container 67b1b17bc45171557ebaf155f472af7e3516ba20ded8ca1ece3c28ecc5615fc4: Status 404 returned error can't find the container with id 67b1b17bc45171557ebaf155f472af7e3516ba20ded8ca1ece3c28ecc5615fc4
Jul 28 20:27:17.772605 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.772459    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b" event={"ID":"4132d533-d2c6-4aeb-a76d-2467f201317a","Type":"ContainerStarted","Data":"9f45ede7e9fb9417f87f265c8e067c8dbd4a33b55a7bbb81d0e496822bb863eb"}
Jul 28 20:27:17.772605 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.772580    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b" event={"ID":"4132d533-d2c6-4aeb-a76d-2467f201317a","Type":"ContainerStarted","Data":"67b1b17bc45171557ebaf155f472af7e3516ba20ded8ca1ece3c28ecc5615fc4"}
Jul 28 20:27:17.772791 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.772613    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b"
Jul 28 20:27:17.774477 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.774359    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp" event={"ID":"f010ebe6-cfad-4f18-8bcf-9f68b3e924ff","Type":"ContainerStarted","Data":"48b9ac673e9b730751430a77aff6e80581915bff2acc47240af3fd3679d70707"}
Jul 28 20:27:17.774477 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.774415    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp" event={"ID":"f010ebe6-cfad-4f18-8bcf-9f68b3e924ff","Type":"ContainerStarted","Data":"292d1909067454f9cbd1c6fb5babf0c4272a91382a0faf9925d7cbecac511bba"}
Jul 28 20:27:17.774477 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.774447    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp"
Jul 28 20:27:17.775770 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.775744    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v" event={"ID":"f7ccdb8a-4a89-439a-91a0-615677e49ee3","Type":"ContainerStarted","Data":"42afdc014923a32879d88fc91d098f82f266ada8c1101d1c7de98378caa33911"}
Jul 28 20:27:17.791603 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.791553    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b" podStartSLOduration=0.791534173 podStartE2EDuration="791.534173ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:27:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:27:17.788112948 +0000 UTC m=+3463.375775892" watchObservedRunningTime="2026-07-28 20:27:17.791534173 +0000 UTC m=+3463.379197102"
Jul 28 20:27:17.813737 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:17.813690    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp" podStartSLOduration=0.813678796 podStartE2EDuration="813.678796ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:27:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:27:17.811772028 +0000 UTC m=+3463.399434973" watchObservedRunningTime="2026-07-28 20:27:17.813678796 +0000 UTC m=+3463.401341774"
Jul 28 20:27:18.783656 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:18.783619    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v" event={"ID":"f7ccdb8a-4a89-439a-91a0-615677e49ee3","Type":"ContainerStarted","Data":"a9facf181361a4c368fc2d4c264d4bfdb5ff4b59652fa5a45214d4a054441f9a"}
Jul 28 20:27:18.784150 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:18.783707    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"
Jul 28 20:27:18.802855 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:18.802813    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v" podStartSLOduration=2.417250493 podStartE2EDuration="2.802800963s" totalImagesPullingTime="385.55047ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:27:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:27:18.801826261 +0000 UTC m=+3464.389489243" watchObservedRunningTime="2026-07-28 20:27:18.802800963 +0000 UTC m=+3464.390463909"
Jul 28 20:27:24.795211 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:24.795175    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"
Jul 28 20:27:28.785776 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:28.785737    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp"
Jul 28 20:27:28.786332 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:28.786136    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b"
Jul 28 20:27:35.819614 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:35.819577    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:35.875538 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:35.875500    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:41.106696 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:41.106662    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:48.015406 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:27:48.015377    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp_f010ebe6-cfad-4f18-8bcf-9f68b3e924ff/payload-processing/0.log"
Jul 28 20:28:00.197622 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.197543    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"]
Jul 28 20:28:00.201164 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.201145    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.203814 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.203789    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-06df9a\"/\"kube-root-ca.crt\""
Jul 28 20:28:00.203941 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.203871    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-06df9a\"/\"default-dockercfg-w9429\""
Jul 28 20:28:00.205118 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.205102    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-06df9a\"/\"ipp-route-06df9a-kserve-self-signed-certs\""
Jul 28 20:28:00.205196 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.205159    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-06df9a\"/\"openshift-service-ca.crt\""
Jul 28 20:28:00.210736 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.210710    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"]
Jul 28 20:28:00.227279 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.227254    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:00.345726 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.345696    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcm57\" (UniqueName: \"kubernetes.io/projected/c6deae6b-d539-42a8-a762-1a0368644b5d-kube-api-access-zcm57\") pod \"ipp-route-06df9a-kserve-56c858f48c-jqsbh\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") " pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.345844 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.345731    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-dshm\") pod \"ipp-route-06df9a-kserve-56c858f48c-jqsbh\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") " pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.345844 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.345779    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-home\") pod \"ipp-route-06df9a-kserve-56c858f48c-jqsbh\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") " pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.345844 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.345815    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6deae6b-d539-42a8-a762-1a0368644b5d-tls-certs\") pod \"ipp-route-06df9a-kserve-56c858f48c-jqsbh\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") " pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.345844 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.345836    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-tmp-dir\") pod \"ipp-route-06df9a-kserve-56c858f48c-jqsbh\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") " pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.345985 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.345905    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-model-cache\") pod \"ipp-route-06df9a-kserve-56c858f48c-jqsbh\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") " pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.446540 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.446516    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcm57\" (UniqueName: \"kubernetes.io/projected/c6deae6b-d539-42a8-a762-1a0368644b5d-kube-api-access-zcm57\") pod \"ipp-route-06df9a-kserve-56c858f48c-jqsbh\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") " pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.446650 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.446547    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-dshm\") pod \"ipp-route-06df9a-kserve-56c858f48c-jqsbh\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") " pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.446650 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.446566    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-home\") pod \"ipp-route-06df9a-kserve-56c858f48c-jqsbh\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") " pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.446650 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.446610    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6deae6b-d539-42a8-a762-1a0368644b5d-tls-certs\") pod \"ipp-route-06df9a-kserve-56c858f48c-jqsbh\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") " pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.446650 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.446632    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-tmp-dir\") pod \"ipp-route-06df9a-kserve-56c858f48c-jqsbh\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") " pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.446872 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.446656    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-model-cache\") pod \"ipp-route-06df9a-kserve-56c858f48c-jqsbh\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") " pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.446997 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.446980    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-home\") pod \"ipp-route-06df9a-kserve-56c858f48c-jqsbh\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") " pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.447104 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.447036    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-tmp-dir\") pod \"ipp-route-06df9a-kserve-56c858f48c-jqsbh\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") " pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.447104 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.447052    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-model-cache\") pod \"ipp-route-06df9a-kserve-56c858f48c-jqsbh\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") " pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.448872 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.448825    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-dshm\") pod \"ipp-route-06df9a-kserve-56c858f48c-jqsbh\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") " pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.449142 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.449126    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6deae6b-d539-42a8-a762-1a0368644b5d-tls-certs\") pod \"ipp-route-06df9a-kserve-56c858f48c-jqsbh\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") " pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.457551 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.457528    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcm57\" (UniqueName: \"kubernetes.io/projected/c6deae6b-d539-42a8-a762-1a0368644b5d-kube-api-access-zcm57\") pod \"ipp-route-06df9a-kserve-56c858f48c-jqsbh\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") " pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.512347 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.512327    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:00.650305 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:00.650228    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"]
Jul 28 20:28:00.653665 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:28:00.653636    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6deae6b_d539_42a8_a762_1a0368644b5d.slice/crio-03ad6d2209658519ce082cc98507503d161311d779d3bba70af4b8564b53912c WatchSource:0}: Error finding container 03ad6d2209658519ce082cc98507503d161311d779d3bba70af4b8564b53912c: Status 404 returned error can't find the container with id 03ad6d2209658519ce082cc98507503d161311d779d3bba70af4b8564b53912c
Jul 28 20:28:01.029125 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:01.029014    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh" event={"ID":"c6deae6b-d539-42a8-a762-1a0368644b5d","Type":"ContainerStarted","Data":"31fd2a99c2a217cc253f839340b8c4e29682db40b538c57dcf6eb4bc5a00061e"}
Jul 28 20:28:01.029125 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:01.029059    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh" event={"ID":"c6deae6b-d539-42a8-a762-1a0368644b5d","Type":"ContainerStarted","Data":"03ad6d2209658519ce082cc98507503d161311d779d3bba70af4b8564b53912c"}
Jul 28 20:28:01.048492 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:01.048446    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh" podStartSLOduration=1.048430212 podStartE2EDuration="1.048430212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:28:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:28:01.046770574 +0000 UTC m=+3506.634433521" watchObservedRunningTime="2026-07-28 20:28:01.048430212 +0000 UTC m=+3506.636093160"
Jul 28 20:28:10.512564 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:10.512520    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:10.513011 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:10.512580    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:10.525143 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:10.525111    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:11.097889 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:11.097853    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:28:43.215504 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:43.215469    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp_f010ebe6-cfad-4f18-8bcf-9f68b3e924ff/payload-processing/0.log"
Jul 28 20:28:43.475894 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:43.475817    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:43.797553 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:43.797479    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:43.895225 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:43.895196    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:46.731107 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:46.731051    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:46.848435 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:46.848390    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:46.864552 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:46.864527    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"]
Jul 28 20:28:46.868058 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:46.868038    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"
Jul 28 20:28:46.871121 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:46.871099    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-0fc004-serving-cert\""
Jul 28 20:28:46.882424 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:46.882400    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"]
Jul 28 20:28:46.916538 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:46.916510    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dcd1c62-c829-4189-a6ea-8c9e8773978d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4\" (UID: \"6dcd1c62-c829-4189-a6ea-8c9e8773978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"
Jul 28 20:28:46.916664 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:46.916587    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcd1c62-c829-4189-a6ea-8c9e8773978d-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4\" (UID: \"6dcd1c62-c829-4189-a6ea-8c9e8773978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"
Jul 28 20:28:46.916664 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:46.916614    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsmrs\" (UniqueName: \"kubernetes.io/projected/6dcd1c62-c829-4189-a6ea-8c9e8773978d-kube-api-access-bsmrs\") pod \"maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4\" (UID: \"6dcd1c62-c829-4189-a6ea-8c9e8773978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"
Jul 28 20:28:46.996107 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:46.996015    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:47.017656 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:47.017630    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dcd1c62-c829-4189-a6ea-8c9e8773978d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4\" (UID: \"6dcd1c62-c829-4189-a6ea-8c9e8773978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"
Jul 28 20:28:47.017757 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:47.017677    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcd1c62-c829-4189-a6ea-8c9e8773978d-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4\" (UID: \"6dcd1c62-c829-4189-a6ea-8c9e8773978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"
Jul 28 20:28:47.017757 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:47.017701    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsmrs\" (UniqueName: \"kubernetes.io/projected/6dcd1c62-c829-4189-a6ea-8c9e8773978d-kube-api-access-bsmrs\") pod \"maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4\" (UID: \"6dcd1c62-c829-4189-a6ea-8c9e8773978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"
Jul 28 20:28:47.017876 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:28:47.017842    2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-serving-cert: secret "maas-api-e2e-ipp-cleanup-0fc004-serving-cert" not found
Jul 28 20:28:47.017934 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:28:47.017928    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dcd1c62-c829-4189-a6ea-8c9e8773978d-maas-api-tls podName:6dcd1c62-c829-4189-a6ea-8c9e8773978d nodeName:}" failed. No retries permitted until 2026-07-28 20:28:47.517907813 +0000 UTC m=+3553.105570751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6dcd1c62-c829-4189-a6ea-8c9e8773978d-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4" (UID: "6dcd1c62-c829-4189-a6ea-8c9e8773978d") : secret "maas-api-e2e-ipp-cleanup-0fc004-serving-cert" not found
Jul 28 20:28:47.018231 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:47.018212    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcd1c62-c829-4189-a6ea-8c9e8773978d-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4\" (UID: \"6dcd1c62-c829-4189-a6ea-8c9e8773978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"
Jul 28 20:28:47.031975 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:47.031946    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsmrs\" (UniqueName: \"kubernetes.io/projected/6dcd1c62-c829-4189-a6ea-8c9e8773978d-kube-api-access-bsmrs\") pod \"maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4\" (UID: \"6dcd1c62-c829-4189-a6ea-8c9e8773978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"
Jul 28 20:28:47.353397 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:47.353374    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:47.521385 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:47.521355    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dcd1c62-c829-4189-a6ea-8c9e8773978d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4\" (UID: \"6dcd1c62-c829-4189-a6ea-8c9e8773978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"
Jul 28 20:28:47.523737 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:47.523708    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dcd1c62-c829-4189-a6ea-8c9e8773978d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4\" (UID: \"6dcd1c62-c829-4189-a6ea-8c9e8773978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"
Jul 28 20:28:47.623873 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:47.623817    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:47.738218 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:47.738189    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:47.778235 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:47.778206    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"
Jul 28 20:28:47.929265 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:47.929159    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"]
Jul 28 20:28:47.932434 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:28:47.932409    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dcd1c62_c829_4189_a6ea_8c9e8773978d.slice/crio-6bb588eb06cbe49af1f8954ddcd8fa6eb0c6ff7622b1c3ba1576993b483f77c0 WatchSource:0}: Error finding container 6bb588eb06cbe49af1f8954ddcd8fa6eb0c6ff7622b1c3ba1576993b483f77c0: Status 404 returned error can't find the container with id 6bb588eb06cbe49af1f8954ddcd8fa6eb0c6ff7622b1c3ba1576993b483f77c0
Jul 28 20:28:48.303376 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:48.303301    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4" event={"ID":"6dcd1c62-c829-4189-a6ea-8c9e8773978d","Type":"ContainerStarted","Data":"6bb588eb06cbe49af1f8954ddcd8fa6eb0c6ff7622b1c3ba1576993b483f77c0"}
Jul 28 20:28:49.310349 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:49.310316    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4" event={"ID":"6dcd1c62-c829-4189-a6ea-8c9e8773978d","Type":"ContainerStarted","Data":"ea50293caca15bc6df0ca4db6aaaf4d56cfe6aff452c97750872d8ecca1bbd63"}
Jul 28 20:28:49.310740 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:49.310441    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"
Jul 28 20:28:49.328449 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:49.328403    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4" podStartSLOduration=2.753309366 podStartE2EDuration="3.32838684s" totalImagesPullingTime="575.077474ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:28:49.326900408 +0000 UTC m=+3554.914563353" watchObservedRunningTime="2026-07-28 20:28:49.32838684 +0000 UTC m=+3554.916049801"
Jul 28 20:28:55.321420 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:28:55.321390    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"
Jul 28 20:29:11.718566 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:11.718528    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:17.604794 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:17.604758    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"]
Jul 28 20:29:17.605199 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:17.604969    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4" podUID="6dcd1c62-c829-4189-a6ea-8c9e8773978d" containerName="maas-api" containerID="cri-o://ea50293caca15bc6df0ca4db6aaaf4d56cfe6aff452c97750872d8ecca1bbd63" gracePeriod=30
Jul 28 20:29:17.634030 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:17.634002    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:17.714937 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:17.714909    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:17.786223 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:17.786195    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:17.857309 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:17.857250    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"
Jul 28 20:29:17.966648 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:17.966623    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcd1c62-c829-4189-a6ea-8c9e8773978d-openshift-service-ca\") pod \"6dcd1c62-c829-4189-a6ea-8c9e8773978d\" (UID: \"6dcd1c62-c829-4189-a6ea-8c9e8773978d\") "
Jul 28 20:29:17.966809 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:17.966664    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dcd1c62-c829-4189-a6ea-8c9e8773978d-maas-api-tls\") pod \"6dcd1c62-c829-4189-a6ea-8c9e8773978d\" (UID: \"6dcd1c62-c829-4189-a6ea-8c9e8773978d\") "
Jul 28 20:29:17.966809 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:17.966744    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsmrs\" (UniqueName: \"kubernetes.io/projected/6dcd1c62-c829-4189-a6ea-8c9e8773978d-kube-api-access-bsmrs\") pod \"6dcd1c62-c829-4189-a6ea-8c9e8773978d\" (UID: \"6dcd1c62-c829-4189-a6ea-8c9e8773978d\") "
Jul 28 20:29:17.966988 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:17.966966    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dcd1c62-c829-4189-a6ea-8c9e8773978d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6dcd1c62-c829-4189-a6ea-8c9e8773978d" (UID: "6dcd1c62-c829-4189-a6ea-8c9e8773978d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:29:17.967116 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:17.967096    2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcd1c62-c829-4189-a6ea-8c9e8773978d-openshift-service-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:29:17.968988 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:17.968962    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dcd1c62-c829-4189-a6ea-8c9e8773978d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6dcd1c62-c829-4189-a6ea-8c9e8773978d" (UID: "6dcd1c62-c829-4189-a6ea-8c9e8773978d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:29:17.969094 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:17.969029    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dcd1c62-c829-4189-a6ea-8c9e8773978d-kube-api-access-bsmrs" (OuterVolumeSpecName: "kube-api-access-bsmrs") pod "6dcd1c62-c829-4189-a6ea-8c9e8773978d" (UID: "6dcd1c62-c829-4189-a6ea-8c9e8773978d"). InnerVolumeSpecName "kube-api-access-bsmrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:29:18.067718 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:18.067694    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dcd1c62-c829-4189-a6ea-8c9e8773978d-maas-api-tls\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:29:18.067718 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:18.067716    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsmrs\" (UniqueName: \"kubernetes.io/projected/6dcd1c62-c829-4189-a6ea-8c9e8773978d-kube-api-access-bsmrs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:29:18.486043 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:18.486018    2572 generic.go:356] "Generic (PLEG): container finished" podID="6dcd1c62-c829-4189-a6ea-8c9e8773978d" containerID="ea50293caca15bc6df0ca4db6aaaf4d56cfe6aff452c97750872d8ecca1bbd63" exitCode=0
Jul 28 20:29:18.486161 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:18.486060    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"
Jul 28 20:29:18.486161 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:18.486095    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4" event={"ID":"6dcd1c62-c829-4189-a6ea-8c9e8773978d","Type":"ContainerDied","Data":"ea50293caca15bc6df0ca4db6aaaf4d56cfe6aff452c97750872d8ecca1bbd63"}
Jul 28 20:29:18.486161 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:18.486147    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4" event={"ID":"6dcd1c62-c829-4189-a6ea-8c9e8773978d","Type":"ContainerDied","Data":"6bb588eb06cbe49af1f8954ddcd8fa6eb0c6ff7622b1c3ba1576993b483f77c0"}
Jul 28 20:29:18.486269 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:18.486171    2572 scope.go:117] "RemoveContainer" containerID="ea50293caca15bc6df0ca4db6aaaf4d56cfe6aff452c97750872d8ecca1bbd63"
Jul 28 20:29:18.495328 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:18.495311    2572 scope.go:117] "RemoveContainer" containerID="ea50293caca15bc6df0ca4db6aaaf4d56cfe6aff452c97750872d8ecca1bbd63"
Jul 28 20:29:18.495584 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:29:18.495565    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea50293caca15bc6df0ca4db6aaaf4d56cfe6aff452c97750872d8ecca1bbd63\": container with ID starting with ea50293caca15bc6df0ca4db6aaaf4d56cfe6aff452c97750872d8ecca1bbd63 not found: ID does not exist" containerID="ea50293caca15bc6df0ca4db6aaaf4d56cfe6aff452c97750872d8ecca1bbd63"
Jul 28 20:29:18.495650 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:18.495590    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea50293caca15bc6df0ca4db6aaaf4d56cfe6aff452c97750872d8ecca1bbd63"} err="failed to get container status \"ea50293caca15bc6df0ca4db6aaaf4d56cfe6aff452c97750872d8ecca1bbd63\": rpc error: code = NotFound desc = could not find container \"ea50293caca15bc6df0ca4db6aaaf4d56cfe6aff452c97750872d8ecca1bbd63\": container with ID starting with ea50293caca15bc6df0ca4db6aaaf4d56cfe6aff452c97750872d8ecca1bbd63 not found: ID does not exist"
Jul 28 20:29:18.508235 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:18.508216    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"]
Jul 28 20:29:18.513861 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:18.513840    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0fc004-75d9b88f74-z9jf4"]
Jul 28 20:29:18.605644 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:18.605621    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:18.685603 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:18.685577    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:18.864536 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:18.864506    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dcd1c62-c829-4189-a6ea-8c9e8773978d" path="/var/lib/kubelet/pods/6dcd1c62-c829-4189-a6ea-8c9e8773978d/volumes"
Jul 28 20:29:24.199897 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:24.199857    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:25.652764 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:25.652735    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:28.054903 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:28.054790    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-06df9a-84krq"]
Jul 28 20:29:28.055472 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:28.055446    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dcd1c62-c829-4189-a6ea-8c9e8773978d" containerName="maas-api"
Jul 28 20:29:28.055570 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:28.055475    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dcd1c62-c829-4189-a6ea-8c9e8773978d" containerName="maas-api"
Jul 28 20:29:28.055649 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:28.055632    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dcd1c62-c829-4189-a6ea-8c9e8773978d" containerName="maas-api"
Jul 28 20:29:28.058784 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:28.058759    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-06df9a-84krq"
Jul 28 20:29:28.061406 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:28.061381    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6zs4\""
Jul 28 20:29:28.066328 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:28.066300    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-06df9a-84krq"]
Jul 28 20:29:28.155541 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:28.155516    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e9820f-02ea-4ea4-83fb-b284bf3b7fee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-06df9a-84krq\" (UID: \"29e9820f-02ea-4ea4-83fb-b284bf3b7fee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-06df9a-84krq"
Jul 28 20:29:28.256742 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:28.256714    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e9820f-02ea-4ea4-83fb-b284bf3b7fee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-06df9a-84krq\" (UID: \"29e9820f-02ea-4ea4-83fb-b284bf3b7fee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-06df9a-84krq"
Jul 28 20:29:28.257306 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:28.257288    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e9820f-02ea-4ea4-83fb-b284bf3b7fee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-06df9a-84krq\" (UID: \"29e9820f-02ea-4ea4-83fb-b284bf3b7fee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-06df9a-84krq"
Jul 28 20:29:28.372985 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:28.372964    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-06df9a-84krq"
Jul 28 20:29:28.509442 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:28.509415    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-06df9a-84krq"]
Jul 28 20:29:28.511517 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:29:28.511481    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29e9820f_02ea_4ea4_83fb_b284bf3b7fee.slice/crio-1e0f4d2d15f8fefa42b4806a6b640849dee5b2a0315b5c19c0767d3d384d3c0b WatchSource:0}: Error finding container 1e0f4d2d15f8fefa42b4806a6b640849dee5b2a0315b5c19c0767d3d384d3c0b: Status 404 returned error can't find the container with id 1e0f4d2d15f8fefa42b4806a6b640849dee5b2a0315b5c19c0767d3d384d3c0b
Jul 28 20:29:28.552564 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:28.552535    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-06df9a-84krq" event={"ID":"29e9820f-02ea-4ea4-83fb-b284bf3b7fee","Type":"ContainerStarted","Data":"1e0f4d2d15f8fefa42b4806a6b640849dee5b2a0315b5c19c0767d3d384d3c0b"}
Jul 28 20:29:29.560130 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:29.560096    2572 generic.go:356] "Generic (PLEG): container finished" podID="29e9820f-02ea-4ea4-83fb-b284bf3b7fee" containerID="f3f53ff829472d04fac1ef7812c6b65ba68e682e13f2c876d84385c2dd1b3f8f" exitCode=0
Jul 28 20:29:29.560523 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:29.560185    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-06df9a-84krq" event={"ID":"29e9820f-02ea-4ea4-83fb-b284bf3b7fee","Type":"ContainerDied","Data":"f3f53ff829472d04fac1ef7812c6b65ba68e682e13f2c876d84385c2dd1b3f8f"}
Jul 28 20:29:30.693021 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:30.692999    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-06df9a-84krq"
Jul 28 20:29:30.774369 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:30.774345    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e9820f-02ea-4ea4-83fb-b284bf3b7fee-maas-api-service-ca\") pod \"29e9820f-02ea-4ea4-83fb-b284bf3b7fee\" (UID: \"29e9820f-02ea-4ea4-83fb-b284bf3b7fee\") "
Jul 28 20:29:30.774650 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:30.774631    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29e9820f-02ea-4ea4-83fb-b284bf3b7fee-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "29e9820f-02ea-4ea4-83fb-b284bf3b7fee" (UID: "29e9820f-02ea-4ea4-83fb-b284bf3b7fee"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:29:30.875234 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:30.875216    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e9820f-02ea-4ea4-83fb-b284bf3b7fee-maas-api-service-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:29:31.572531 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:31.572496    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-06df9a-84krq" event={"ID":"29e9820f-02ea-4ea4-83fb-b284bf3b7fee","Type":"ContainerDied","Data":"1e0f4d2d15f8fefa42b4806a6b640849dee5b2a0315b5c19c0767d3d384d3c0b"}
Jul 28 20:29:31.572678 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:31.572534    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-06df9a-84krq"
Jul 28 20:29:31.572678 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:31.572552    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e0f4d2d15f8fefa42b4806a6b640849dee5b2a0315b5c19c0767d3d384d3c0b"
Jul 28 20:29:35.111964 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:35.111935    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 20:29:35.120274 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:35.120249    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 20:29:43.674016 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:43.673980    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:43.744905 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:43.744862    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:43.850106 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:43.850054    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-06df9a-84krq"]
Jul 28 20:29:43.852425 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:43.852406    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-06df9a-84krq"]
Jul 28 20:29:43.899780 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:43.899754    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:43.956726 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:43.956650    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:44.197324 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:44.197293    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:44.864224 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:44.864177    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29e9820f-02ea-4ea4-83fb-b284bf3b7fee" path="/var/lib/kubelet/pods/29e9820f-02ea-4ea4-83fb-b284bf3b7fee/volumes"
Jul 28 20:29:49.183365 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:49.183324    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"]
Jul 28 20:29:49.239359 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:29:49.239323    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:04.203007 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.202953    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh" podUID="c6deae6b-d539-42a8-a762-1a0368644b5d" containerName="main" containerID="cri-o://31fd2a99c2a217cc253f839340b8c4e29682db40b538c57dcf6eb4bc5a00061e" gracePeriod=45
Jul 28 20:30:04.446349 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.446327    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:30:04.539293 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.539222    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-dshm\") pod \"c6deae6b-d539-42a8-a762-1a0368644b5d\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") "
Jul 28 20:30:04.539293 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.539276    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-model-cache\") pod \"c6deae6b-d539-42a8-a762-1a0368644b5d\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") "
Jul 28 20:30:04.539485 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.539315    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-tmp-dir\") pod \"c6deae6b-d539-42a8-a762-1a0368644b5d\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") "
Jul 28 20:30:04.539485 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.539333    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-home\") pod \"c6deae6b-d539-42a8-a762-1a0368644b5d\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") "
Jul 28 20:30:04.539485 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.539372    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zcm57\" (UniqueName: \"kubernetes.io/projected/c6deae6b-d539-42a8-a762-1a0368644b5d-kube-api-access-zcm57\") pod \"c6deae6b-d539-42a8-a762-1a0368644b5d\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") "
Jul 28 20:30:04.539485 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.539406    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6deae6b-d539-42a8-a762-1a0368644b5d-tls-certs\") pod \"c6deae6b-d539-42a8-a762-1a0368644b5d\" (UID: \"c6deae6b-d539-42a8-a762-1a0368644b5d\") "
Jul 28 20:30:04.539712 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.539593    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c6deae6b-d539-42a8-a762-1a0368644b5d" (UID: "c6deae6b-d539-42a8-a762-1a0368644b5d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:30:04.539712 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.539609    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-model-cache" (OuterVolumeSpecName: "model-cache") pod "c6deae6b-d539-42a8-a762-1a0368644b5d" (UID: "c6deae6b-d539-42a8-a762-1a0368644b5d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:30:04.539712 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.539620    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-home" (OuterVolumeSpecName: "home") pod "c6deae6b-d539-42a8-a762-1a0368644b5d" (UID: "c6deae6b-d539-42a8-a762-1a0368644b5d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:30:04.539844 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.539735    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-model-cache\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:30:04.539844 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.539752    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-tmp-dir\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:30:04.539844 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.539763    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-home\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:30:04.541543 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.541517    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6deae6b-d539-42a8-a762-1a0368644b5d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c6deae6b-d539-42a8-a762-1a0368644b5d" (UID: "c6deae6b-d539-42a8-a762-1a0368644b5d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:30:04.541618 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.541605    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6deae6b-d539-42a8-a762-1a0368644b5d-kube-api-access-zcm57" (OuterVolumeSpecName: "kube-api-access-zcm57") pod "c6deae6b-d539-42a8-a762-1a0368644b5d" (UID: "c6deae6b-d539-42a8-a762-1a0368644b5d"). InnerVolumeSpecName "kube-api-access-zcm57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:30:04.541665 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.541634    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-dshm" (OuterVolumeSpecName: "dshm") pod "c6deae6b-d539-42a8-a762-1a0368644b5d" (UID: "c6deae6b-d539-42a8-a762-1a0368644b5d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:30:04.640479 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.640445    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcm57\" (UniqueName: \"kubernetes.io/projected/c6deae6b-d539-42a8-a762-1a0368644b5d-kube-api-access-zcm57\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:30:04.640479 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.640478    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6deae6b-d539-42a8-a762-1a0368644b5d-tls-certs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:30:04.640479 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.640491    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6deae6b-d539-42a8-a762-1a0368644b5d-dshm\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:30:04.776558 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.776535    2572 generic.go:356] "Generic (PLEG): container finished" podID="c6deae6b-d539-42a8-a762-1a0368644b5d" containerID="31fd2a99c2a217cc253f839340b8c4e29682db40b538c57dcf6eb4bc5a00061e" exitCode=0
Jul 28 20:30:04.776666 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.776566    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh" event={"ID":"c6deae6b-d539-42a8-a762-1a0368644b5d","Type":"ContainerDied","Data":"31fd2a99c2a217cc253f839340b8c4e29682db40b538c57dcf6eb4bc5a00061e"}
Jul 28 20:30:04.776666 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.776572    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"
Jul 28 20:30:04.776666 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.776594    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh" event={"ID":"c6deae6b-d539-42a8-a762-1a0368644b5d","Type":"ContainerDied","Data":"03ad6d2209658519ce082cc98507503d161311d779d3bba70af4b8564b53912c"}
Jul 28 20:30:04.776666 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.776619    2572 scope.go:117] "RemoveContainer" containerID="31fd2a99c2a217cc253f839340b8c4e29682db40b538c57dcf6eb4bc5a00061e"
Jul 28 20:30:04.794267 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.794245    2572 scope.go:117] "RemoveContainer" containerID="31fd2a99c2a217cc253f839340b8c4e29682db40b538c57dcf6eb4bc5a00061e"
Jul 28 20:30:04.794499 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:30:04.794480    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31fd2a99c2a217cc253f839340b8c4e29682db40b538c57dcf6eb4bc5a00061e\": container with ID starting with 31fd2a99c2a217cc253f839340b8c4e29682db40b538c57dcf6eb4bc5a00061e not found: ID does not exist" containerID="31fd2a99c2a217cc253f839340b8c4e29682db40b538c57dcf6eb4bc5a00061e"
Jul 28 20:30:04.794555 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.794507    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31fd2a99c2a217cc253f839340b8c4e29682db40b538c57dcf6eb4bc5a00061e"} err="failed to get container status \"31fd2a99c2a217cc253f839340b8c4e29682db40b538c57dcf6eb4bc5a00061e\": rpc error: code = NotFound desc = could not find container \"31fd2a99c2a217cc253f839340b8c4e29682db40b538c57dcf6eb4bc5a00061e\": container with ID starting with 31fd2a99c2a217cc253f839340b8c4e29682db40b538c57dcf6eb4bc5a00061e not found: ID does not exist"
Jul 28 20:30:04.802324 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.802302    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"]
Jul 28 20:30:04.807255 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.807235    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-06df9a/ipp-route-06df9a-kserve-56c858f48c-jqsbh"]
Jul 28 20:30:04.864514 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:04.864484    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6deae6b-d539-42a8-a762-1a0368644b5d" path="/var/lib/kubelet/pods/c6deae6b-d539-42a8-a762-1a0368644b5d/volumes"
Jul 28 20:30:26.123061 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:26.123028    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:26.479945 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:26.479859    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:32.042139 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:32.042093    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:32.139487 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:32.139450    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:32.308314 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:32.308279    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:32.536089 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:32.536039    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:43.160737 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.160704    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"]
Jul 28 20:30:43.161174 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.160881    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v" podUID="f7ccdb8a-4a89-439a-91a0-615677e49ee3" containerName="maas-api" containerID="cri-o://a9facf181361a4c368fc2d4c264d4bfdb5ff4b59652fa5a45214d4a054441f9a" gracePeriod=30
Jul 28 20:30:43.189663 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.189637    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:43.214019 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.213990    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp"]
Jul 28 20:30:43.214269 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.214242    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp" podUID="f010ebe6-cfad-4f18-8bcf-9f68b3e924ff" containerName="payload-processing" containerID="cri-o://48b9ac673e9b730751430a77aff6e80581915bff2acc47240af3fd3679d70707" gracePeriod=30
Jul 28 20:30:43.222751 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.222724    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b"]
Jul 28 20:30:43.222951 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.222922    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b" podUID="4132d533-d2c6-4aeb-a76d-2467f201317a" containerName="payload-pre-processing" containerID="cri-o://9f45ede7e9fb9417f87f265c8e067c8dbd4a33b55a7bbb81d0e496822bb863eb" gracePeriod=30
Jul 28 20:30:43.256642 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.256612    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:43.356852 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.356797    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:43.435093 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.435019    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:43.507579 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.507554    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"
Jul 28 20:30:43.509513 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.509495    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp"
Jul 28 20:30:43.528566 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.528479    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b"
Jul 28 20:30:43.573992 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.573968    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7ccdb8a-4a89-439a-91a0-615677e49ee3-openshift-service-ca\") pod \"f7ccdb8a-4a89-439a-91a0-615677e49ee3\" (UID: \"f7ccdb8a-4a89-439a-91a0-615677e49ee3\") "
Jul 28 20:30:43.574158 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.574010    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26276\" (UniqueName: \"kubernetes.io/projected/f010ebe6-cfad-4f18-8bcf-9f68b3e924ff-kube-api-access-26276\") pod \"f010ebe6-cfad-4f18-8bcf-9f68b3e924ff\" (UID: \"f010ebe6-cfad-4f18-8bcf-9f68b3e924ff\") "
Jul 28 20:30:43.574158 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.574034    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2gkd\" (UniqueName: \"kubernetes.io/projected/f7ccdb8a-4a89-439a-91a0-615677e49ee3-kube-api-access-p2gkd\") pod \"f7ccdb8a-4a89-439a-91a0-615677e49ee3\" (UID: \"f7ccdb8a-4a89-439a-91a0-615677e49ee3\") "
Jul 28 20:30:43.574158 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.574061    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f010ebe6-cfad-4f18-8bcf-9f68b3e924ff-plugins-config-volume\") pod \"f010ebe6-cfad-4f18-8bcf-9f68b3e924ff\" (UID: \"f010ebe6-cfad-4f18-8bcf-9f68b3e924ff\") "
Jul 28 20:30:43.574386 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.574213    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7ccdb8a-4a89-439a-91a0-615677e49ee3-maas-api-tls\") pod \"f7ccdb8a-4a89-439a-91a0-615677e49ee3\" (UID: \"f7ccdb8a-4a89-439a-91a0-615677e49ee3\") "
Jul 28 20:30:43.574386 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.574278    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbfk8\" (UniqueName: \"kubernetes.io/projected/4132d533-d2c6-4aeb-a76d-2467f201317a-kube-api-access-bbfk8\") pod \"4132d533-d2c6-4aeb-a76d-2467f201317a\" (UID: \"4132d533-d2c6-4aeb-a76d-2467f201317a\") "
Jul 28 20:30:43.574494 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.574389    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4132d533-d2c6-4aeb-a76d-2467f201317a-plugins-config-volume\") pod \"4132d533-d2c6-4aeb-a76d-2467f201317a\" (UID: \"4132d533-d2c6-4aeb-a76d-2467f201317a\") "
Jul 28 20:30:43.574494 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.574393    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7ccdb8a-4a89-439a-91a0-615677e49ee3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f7ccdb8a-4a89-439a-91a0-615677e49ee3" (UID: "f7ccdb8a-4a89-439a-91a0-615677e49ee3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:30:43.574494 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.574425    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f010ebe6-cfad-4f18-8bcf-9f68b3e924ff-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f010ebe6-cfad-4f18-8bcf-9f68b3e924ff" (UID: "f010ebe6-cfad-4f18-8bcf-9f68b3e924ff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:30:43.574825 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.574713    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f010ebe6-cfad-4f18-8bcf-9f68b3e924ff-plugins-config-volume\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:30:43.574825 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.574739    2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7ccdb8a-4a89-439a-91a0-615677e49ee3-openshift-service-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:30:43.574825 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.574816    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4132d533-d2c6-4aeb-a76d-2467f201317a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4132d533-d2c6-4aeb-a76d-2467f201317a" (UID: "4132d533-d2c6-4aeb-a76d-2467f201317a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:30:43.576389 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.576368    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f010ebe6-cfad-4f18-8bcf-9f68b3e924ff-kube-api-access-26276" (OuterVolumeSpecName: "kube-api-access-26276") pod "f010ebe6-cfad-4f18-8bcf-9f68b3e924ff" (UID: "f010ebe6-cfad-4f18-8bcf-9f68b3e924ff"). InnerVolumeSpecName "kube-api-access-26276". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:30:43.576632 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.576616    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7ccdb8a-4a89-439a-91a0-615677e49ee3-kube-api-access-p2gkd" (OuterVolumeSpecName: "kube-api-access-p2gkd") pod "f7ccdb8a-4a89-439a-91a0-615677e49ee3" (UID: "f7ccdb8a-4a89-439a-91a0-615677e49ee3"). InnerVolumeSpecName "kube-api-access-p2gkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:30:43.576706 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.576628    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7ccdb8a-4a89-439a-91a0-615677e49ee3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f7ccdb8a-4a89-439a-91a0-615677e49ee3" (UID: "f7ccdb8a-4a89-439a-91a0-615677e49ee3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:30:43.576912 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.576891    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4132d533-d2c6-4aeb-a76d-2467f201317a-kube-api-access-bbfk8" (OuterVolumeSpecName: "kube-api-access-bbfk8") pod "4132d533-d2c6-4aeb-a76d-2467f201317a" (UID: "4132d533-d2c6-4aeb-a76d-2467f201317a"). InnerVolumeSpecName "kube-api-access-bbfk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:30:43.675602 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.675548    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4132d533-d2c6-4aeb-a76d-2467f201317a-plugins-config-volume\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:30:43.675602 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.675570    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26276\" (UniqueName: \"kubernetes.io/projected/f010ebe6-cfad-4f18-8bcf-9f68b3e924ff-kube-api-access-26276\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:30:43.675602 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.675579    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2gkd\" (UniqueName: \"kubernetes.io/projected/f7ccdb8a-4a89-439a-91a0-615677e49ee3-kube-api-access-p2gkd\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:30:43.675602 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.675589    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7ccdb8a-4a89-439a-91a0-615677e49ee3-maas-api-tls\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:30:43.675602 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.675597    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbfk8\" (UniqueName: \"kubernetes.io/projected/4132d533-d2c6-4aeb-a76d-2467f201317a-kube-api-access-bbfk8\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:30:43.739313 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.739290    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:43.993060 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.992995    2572 generic.go:356] "Generic (PLEG): container finished" podID="4132d533-d2c6-4aeb-a76d-2467f201317a" containerID="9f45ede7e9fb9417f87f265c8e067c8dbd4a33b55a7bbb81d0e496822bb863eb" exitCode=0
Jul 28 20:30:43.993060 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.993046    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b"
Jul 28 20:30:43.993263 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.993095    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b" event={"ID":"4132d533-d2c6-4aeb-a76d-2467f201317a","Type":"ContainerDied","Data":"9f45ede7e9fb9417f87f265c8e067c8dbd4a33b55a7bbb81d0e496822bb863eb"}
Jul 28 20:30:43.993263 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.993155    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b" event={"ID":"4132d533-d2c6-4aeb-a76d-2467f201317a","Type":"ContainerDied","Data":"67b1b17bc45171557ebaf155f472af7e3516ba20ded8ca1ece3c28ecc5615fc4"}
Jul 28 20:30:43.993263 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.993187    2572 scope.go:117] "RemoveContainer" containerID="9f45ede7e9fb9417f87f265c8e067c8dbd4a33b55a7bbb81d0e496822bb863eb"
Jul 28 20:30:43.994514 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.994490    2572 generic.go:356] "Generic (PLEG): container finished" podID="f010ebe6-cfad-4f18-8bcf-9f68b3e924ff" containerID="48b9ac673e9b730751430a77aff6e80581915bff2acc47240af3fd3679d70707" exitCode=0
Jul 28 20:30:43.994651 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.994528    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp"
Jul 28 20:30:43.994651 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.994562    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp" event={"ID":"f010ebe6-cfad-4f18-8bcf-9f68b3e924ff","Type":"ContainerDied","Data":"48b9ac673e9b730751430a77aff6e80581915bff2acc47240af3fd3679d70707"}
Jul 28 20:30:43.994651 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.994619    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp" event={"ID":"f010ebe6-cfad-4f18-8bcf-9f68b3e924ff","Type":"ContainerDied","Data":"292d1909067454f9cbd1c6fb5babf0c4272a91382a0faf9925d7cbecac511bba"}
Jul 28 20:30:43.996336 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.996307    2572 generic.go:356] "Generic (PLEG): container finished" podID="f7ccdb8a-4a89-439a-91a0-615677e49ee3" containerID="a9facf181361a4c368fc2d4c264d4bfdb5ff4b59652fa5a45214d4a054441f9a" exitCode=0
Jul 28 20:30:43.996482 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.996372    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"
Jul 28 20:30:43.996482 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.996375    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v" event={"ID":"f7ccdb8a-4a89-439a-91a0-615677e49ee3","Type":"ContainerDied","Data":"a9facf181361a4c368fc2d4c264d4bfdb5ff4b59652fa5a45214d4a054441f9a"}
Jul 28 20:30:43.996612 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:43.996499    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v" event={"ID":"f7ccdb8a-4a89-439a-91a0-615677e49ee3","Type":"ContainerDied","Data":"42afdc014923a32879d88fc91d098f82f266ada8c1101d1c7de98378caa33911"}
Jul 28 20:30:44.005414 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:44.005383    2572 scope.go:117] "RemoveContainer" containerID="9f45ede7e9fb9417f87f265c8e067c8dbd4a33b55a7bbb81d0e496822bb863eb"
Jul 28 20:30:44.005808 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:30:44.005692    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f45ede7e9fb9417f87f265c8e067c8dbd4a33b55a7bbb81d0e496822bb863eb\": container with ID starting with 9f45ede7e9fb9417f87f265c8e067c8dbd4a33b55a7bbb81d0e496822bb863eb not found: ID does not exist" containerID="9f45ede7e9fb9417f87f265c8e067c8dbd4a33b55a7bbb81d0e496822bb863eb"
Jul 28 20:30:44.005808 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:44.005727    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f45ede7e9fb9417f87f265c8e067c8dbd4a33b55a7bbb81d0e496822bb863eb"} err="failed to get container status \"9f45ede7e9fb9417f87f265c8e067c8dbd4a33b55a7bbb81d0e496822bb863eb\": rpc error: code = NotFound desc = could not find container \"9f45ede7e9fb9417f87f265c8e067c8dbd4a33b55a7bbb81d0e496822bb863eb\": container with ID starting with 9f45ede7e9fb9417f87f265c8e067c8dbd4a33b55a7bbb81d0e496822bb863eb not found: ID does not exist"
Jul 28 20:30:44.005808 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:44.005749    2572 scope.go:117] "RemoveContainer" containerID="48b9ac673e9b730751430a77aff6e80581915bff2acc47240af3fd3679d70707"
Jul 28 20:30:44.014370 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:44.014342    2572 scope.go:117] "RemoveContainer" containerID="48b9ac673e9b730751430a77aff6e80581915bff2acc47240af3fd3679d70707"
Jul 28 20:30:44.014607 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:30:44.014588    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48b9ac673e9b730751430a77aff6e80581915bff2acc47240af3fd3679d70707\": container with ID starting with 48b9ac673e9b730751430a77aff6e80581915bff2acc47240af3fd3679d70707 not found: ID does not exist" containerID="48b9ac673e9b730751430a77aff6e80581915bff2acc47240af3fd3679d70707"
Jul 28 20:30:44.014664 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:44.014617    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48b9ac673e9b730751430a77aff6e80581915bff2acc47240af3fd3679d70707"} err="failed to get container status \"48b9ac673e9b730751430a77aff6e80581915bff2acc47240af3fd3679d70707\": rpc error: code = NotFound desc = could not find container \"48b9ac673e9b730751430a77aff6e80581915bff2acc47240af3fd3679d70707\": container with ID starting with 48b9ac673e9b730751430a77aff6e80581915bff2acc47240af3fd3679d70707 not found: ID does not exist"
Jul 28 20:30:44.014664 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:44.014636    2572 scope.go:117] "RemoveContainer" containerID="a9facf181361a4c368fc2d4c264d4bfdb5ff4b59652fa5a45214d4a054441f9a"
Jul 28 20:30:44.020438 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:44.020416    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b"]
Jul 28 20:30:44.023279 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:44.023260    2572 scope.go:117] "RemoveContainer" containerID="a9facf181361a4c368fc2d4c264d4bfdb5ff4b59652fa5a45214d4a054441f9a"
Jul 28 20:30:44.023559 ip-10-0-138-37 kubenswrapper[2572]: E0728 20:30:44.023535    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9facf181361a4c368fc2d4c264d4bfdb5ff4b59652fa5a45214d4a054441f9a\": container with ID starting with a9facf181361a4c368fc2d4c264d4bfdb5ff4b59652fa5a45214d4a054441f9a not found: ID does not exist" containerID="a9facf181361a4c368fc2d4c264d4bfdb5ff4b59652fa5a45214d4a054441f9a"
Jul 28 20:30:44.023646 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:44.023567    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9facf181361a4c368fc2d4c264d4bfdb5ff4b59652fa5a45214d4a054441f9a"} err="failed to get container status \"a9facf181361a4c368fc2d4c264d4bfdb5ff4b59652fa5a45214d4a054441f9a\": rpc error: code = NotFound desc = could not find container \"a9facf181361a4c368fc2d4c264d4bfdb5ff4b59652fa5a45214d4a054441f9a\": container with ID starting with a9facf181361a4c368fc2d4c264d4bfdb5ff4b59652fa5a45214d4a054441f9a not found: ID does not exist"
Jul 28 20:30:44.026749 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:44.026724    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-426065-6ddb76d8d-flr6b"]
Jul 28 20:30:44.037633 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:44.037612    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp"]
Jul 28 20:30:44.043822 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:44.043803    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-426065-7596c67c9d-nfbmp"]
Jul 28 20:30:44.053610 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:44.053588    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"]
Jul 28 20:30:44.055553 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:44.055535    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-426065-6d8bc9b5cb-fqm9v"]
Jul 28 20:30:44.865454 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:44.865422    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4132d533-d2c6-4aeb-a76d-2467f201317a" path="/var/lib/kubelet/pods/4132d533-d2c6-4aeb-a76d-2467f201317a/volumes"
Jul 28 20:30:44.865847 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:44.865762    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f010ebe6-cfad-4f18-8bcf-9f68b3e924ff" path="/var/lib/kubelet/pods/f010ebe6-cfad-4f18-8bcf-9f68b3e924ff/volumes"
Jul 28 20:30:44.866136 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:44.866120    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7ccdb8a-4a89-439a-91a0-615677e49ee3" path="/var/lib/kubelet/pods/f7ccdb8a-4a89-439a-91a0-615677e49ee3/volumes"
Jul 28 20:30:49.701171 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:49.701137    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:49.774613 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:49.774567    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:49.973324 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:49.973237    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:50.189054 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.189022    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:50.323994 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.323961    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c38c57-f2wxj"]
Jul 28 20:30:50.324406 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.324388    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29e9820f-02ea-4ea4-83fb-b284bf3b7fee" containerName="revoke-keys"
Jul 28 20:30:50.324493 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.324407    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="29e9820f-02ea-4ea4-83fb-b284bf3b7fee" containerName="revoke-keys"
Jul 28 20:30:50.324493 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.324422    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6deae6b-d539-42a8-a762-1a0368644b5d" containerName="main"
Jul 28 20:30:50.324493 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.324429    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6deae6b-d539-42a8-a762-1a0368644b5d" containerName="main"
Jul 28 20:30:50.324493 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.324444    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7ccdb8a-4a89-439a-91a0-615677e49ee3" containerName="maas-api"
Jul 28 20:30:50.324493 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.324454    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7ccdb8a-4a89-439a-91a0-615677e49ee3" containerName="maas-api"
Jul 28 20:30:50.324493 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.324467    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f010ebe6-cfad-4f18-8bcf-9f68b3e924ff" containerName="payload-processing"
Jul 28 20:30:50.324493 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.324477    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f010ebe6-cfad-4f18-8bcf-9f68b3e924ff" containerName="payload-processing"
Jul 28 20:30:50.324840 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.324499    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4132d533-d2c6-4aeb-a76d-2467f201317a" containerName="payload-pre-processing"
Jul 28 20:30:50.324840 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.324507    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4132d533-d2c6-4aeb-a76d-2467f201317a" containerName="payload-pre-processing"
Jul 28 20:30:50.324840 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.324607    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f010ebe6-cfad-4f18-8bcf-9f68b3e924ff" containerName="payload-processing"
Jul 28 20:30:50.324840 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.324621    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4132d533-d2c6-4aeb-a76d-2467f201317a" containerName="payload-pre-processing"
Jul 28 20:30:50.324840 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.324634    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6deae6b-d539-42a8-a762-1a0368644b5d" containerName="main"
Jul 28 20:30:50.324840 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.324643    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="29e9820f-02ea-4ea4-83fb-b284bf3b7fee" containerName="revoke-keys"
Jul 28 20:30:50.324840 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.324652    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7ccdb8a-4a89-439a-91a0-615677e49ee3" containerName="maas-api"
Jul 28 20:30:50.327685 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.327665    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c38c57-f2wxj"
Jul 28 20:30:50.330272 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.330257    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6zs4\""
Jul 28 20:30:50.335423 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.335183    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c38c57-f2wxj"]
Jul 28 20:30:50.431459 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.431421    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10cba5bf-8d4a-45ab-a968-06310a7f50aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c38c57-f2wxj\" (UID: \"10cba5bf-8d4a-45ab-a968-06310a7f50aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c38c57-f2wxj"
Jul 28 20:30:50.532680 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.532641    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10cba5bf-8d4a-45ab-a968-06310a7f50aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c38c57-f2wxj\" (UID: \"10cba5bf-8d4a-45ab-a968-06310a7f50aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c38c57-f2wxj"
Jul 28 20:30:50.533419 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.533391    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10cba5bf-8d4a-45ab-a968-06310a7f50aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c38c57-f2wxj\" (UID: \"10cba5bf-8d4a-45ab-a968-06310a7f50aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c38c57-f2wxj"
Jul 28 20:30:50.638722 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.638617    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c38c57-f2wxj"
Jul 28 20:30:50.790030 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:50.789991    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c38c57-f2wxj"]
Jul 28 20:30:50.791443 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:30:50.791410    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10cba5bf_8d4a_45ab_a968_06310a7f50aa.slice/crio-66eeb11c0ae926a81e340edd962a6d07eac9e103c94f66cdac0527159f814afd WatchSource:0}: Error finding container 66eeb11c0ae926a81e340edd962a6d07eac9e103c94f66cdac0527159f814afd: Status 404 returned error can't find the container with id 66eeb11c0ae926a81e340edd962a6d07eac9e103c94f66cdac0527159f814afd
Jul 28 20:30:51.039942 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:51.039855    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c38c57-f2wxj" event={"ID":"10cba5bf-8d4a-45ab-a968-06310a7f50aa","Type":"ContainerStarted","Data":"66eeb11c0ae926a81e340edd962a6d07eac9e103c94f66cdac0527159f814afd"}
Jul 28 20:30:52.049642 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:52.049607    2572 generic.go:356] "Generic (PLEG): container finished" podID="10cba5bf-8d4a-45ab-a968-06310a7f50aa" containerID="9ab1939d139146aa078c9ba9051f6888c95c2aa2c090c80b3cdd4e34c54dbe17" exitCode=0
Jul 28 20:30:52.050093 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:52.049661    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c38c57-f2wxj" event={"ID":"10cba5bf-8d4a-45ab-a968-06310a7f50aa","Type":"ContainerDied","Data":"9ab1939d139146aa078c9ba9051f6888c95c2aa2c090c80b3cdd4e34c54dbe17"}
Jul 28 20:30:53.187166 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:53.187139    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c38c57-f2wxj"
Jul 28 20:30:53.257080 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:53.257049    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10cba5bf-8d4a-45ab-a968-06310a7f50aa-maas-api-service-ca\") pod \"10cba5bf-8d4a-45ab-a968-06310a7f50aa\" (UID: \"10cba5bf-8d4a-45ab-a968-06310a7f50aa\") "
Jul 28 20:30:53.257399 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:53.257377    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10cba5bf-8d4a-45ab-a968-06310a7f50aa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "10cba5bf-8d4a-45ab-a968-06310a7f50aa" (UID: "10cba5bf-8d4a-45ab-a968-06310a7f50aa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:30:53.358740 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:53.358654    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10cba5bf-8d4a-45ab-a968-06310a7f50aa-maas-api-service-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:30:54.061174 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:54.061146    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c38c57-f2wxj"
Jul 28 20:30:54.061326 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:54.061193    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c38c57-f2wxj" event={"ID":"10cba5bf-8d4a-45ab-a968-06310a7f50aa","Type":"ContainerDied","Data":"66eeb11c0ae926a81e340edd962a6d07eac9e103c94f66cdac0527159f814afd"}
Jul 28 20:30:54.061326 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:30:54.061239    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66eeb11c0ae926a81e340edd962a6d07eac9e103c94f66cdac0527159f814afd"
Jul 28 20:31:00.849465 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:00.849385    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:00.942216 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:00.942185    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:01.042118 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:01.042088    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:01.056205 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:01.056178    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c38c57-f2wxj"]
Jul 28 20:31:01.060630 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:01.060611    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c38c57-f2wxj"]
Jul 28 20:31:01.117648 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:01.117569    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:01.438312 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:01.438231    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:02.864335 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:02.864293    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10cba5bf-8d4a-45ab-a968-06310a7f50aa" path="/var/lib/kubelet/pods/10cba5bf-8d4a-45ab-a968-06310a7f50aa/volumes"
Jul 28 20:31:07.451988 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:07.451948    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:07.517830 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:07.517800    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:07.729080 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:07.728979    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:07.946546 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:07.946515    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:08.054057 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:08.053982    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr"]
Jul 28 20:31:08.054439 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:08.054423    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10cba5bf-8d4a-45ab-a968-06310a7f50aa" containerName="revoke-keys"
Jul 28 20:31:08.054486 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:08.054441    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="10cba5bf-8d4a-45ab-a968-06310a7f50aa" containerName="revoke-keys"
Jul 28 20:31:08.054527 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:08.054510    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="10cba5bf-8d4a-45ab-a968-06310a7f50aa" containerName="revoke-keys"
Jul 28 20:31:08.057760 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:08.057744    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr"
Jul 28 20:31:08.060432 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:08.060413    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z6zs4\""
Jul 28 20:31:08.066428 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:08.065726    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr"]
Jul 28 20:31:08.187298 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:08.187276    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff1a5731-c7d8-4a1d-9639-102044250eba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr\" (UID: \"ff1a5731-c7d8-4a1d-9639-102044250eba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr"
Jul 28 20:31:08.287983 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:08.287950    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff1a5731-c7d8-4a1d-9639-102044250eba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr\" (UID: \"ff1a5731-c7d8-4a1d-9639-102044250eba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr"
Jul 28 20:31:08.288545 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:08.288527    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff1a5731-c7d8-4a1d-9639-102044250eba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr\" (UID: \"ff1a5731-c7d8-4a1d-9639-102044250eba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr"
Jul 28 20:31:08.370123 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:08.370099    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr"
Jul 28 20:31:08.706531 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:08.706503    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr"]
Jul 28 20:31:08.707997 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:31:08.707970    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff1a5731_c7d8_4a1d_9639_102044250eba.slice/crio-0a375ed5595cc6f4e80048541c7056e50c54b8e8656ea9a567f48cd83980a4b4 WatchSource:0}: Error finding container 0a375ed5595cc6f4e80048541c7056e50c54b8e8656ea9a567f48cd83980a4b4: Status 404 returned error can't find the container with id 0a375ed5595cc6f4e80048541c7056e50c54b8e8656ea9a567f48cd83980a4b4
Jul 28 20:31:09.150341 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:09.150311    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr" event={"ID":"ff1a5731-c7d8-4a1d-9639-102044250eba","Type":"ContainerStarted","Data":"6c5af65a22f82ea0990572d3a3b43e22329e79c5a54477067fc90749f68f85b3"}
Jul 28 20:31:09.150477 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:09.150365    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr" event={"ID":"ff1a5731-c7d8-4a1d-9639-102044250eba","Type":"ContainerStarted","Data":"0a375ed5595cc6f4e80048541c7056e50c54b8e8656ea9a567f48cd83980a4b4"}
Jul 28 20:31:09.171410 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:09.171372    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr" podStartSLOduration=1.171357142 podStartE2EDuration="1.171357142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:31:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:31:09.169795954 +0000 UTC m=+3694.757458910" watchObservedRunningTime="2026-07-28 20:31:09.171357142 +0000 UTC m=+3694.759020088"
Jul 28 20:31:10.159031 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:10.158992    2572 generic.go:356] "Generic (PLEG): container finished" podID="ff1a5731-c7d8-4a1d-9639-102044250eba" containerID="6c5af65a22f82ea0990572d3a3b43e22329e79c5a54477067fc90749f68f85b3" exitCode=0
Jul 28 20:31:10.159475 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:10.159030    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr" event={"ID":"ff1a5731-c7d8-4a1d-9639-102044250eba","Type":"ContainerDied","Data":"6c5af65a22f82ea0990572d3a3b43e22329e79c5a54477067fc90749f68f85b3"}
Jul 28 20:31:11.290483 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:11.290462    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr"
Jul 28 20:31:11.413164 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:11.413141    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff1a5731-c7d8-4a1d-9639-102044250eba-maas-api-service-ca\") pod \"ff1a5731-c7d8-4a1d-9639-102044250eba\" (UID: \"ff1a5731-c7d8-4a1d-9639-102044250eba\") "
Jul 28 20:31:11.413466 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:11.413445    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff1a5731-c7d8-4a1d-9639-102044250eba-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ff1a5731-c7d8-4a1d-9639-102044250eba" (UID: "ff1a5731-c7d8-4a1d-9639-102044250eba"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:31:11.514285 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:11.514230    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff1a5731-c7d8-4a1d-9639-102044250eba-maas-api-service-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\""
Jul 28 20:31:12.171670 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:12.171645    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr"
Jul 28 20:31:12.171838 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:12.171632    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr" event={"ID":"ff1a5731-c7d8-4a1d-9639-102044250eba","Type":"ContainerDied","Data":"0a375ed5595cc6f4e80048541c7056e50c54b8e8656ea9a567f48cd83980a4b4"}
Jul 28 20:31:12.171838 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:12.171792    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a375ed5595cc6f4e80048541c7056e50c54b8e8656ea9a567f48cd83980a4b4"
Jul 28 20:31:18.811502 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:18.811458    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:18.897791 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:18.897762    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:19.015125 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:19.015091    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr"]
Jul 28 20:31:19.020431 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:19.020407    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d170bd-lztqr"]
Jul 28 20:31:19.021275 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:19.021245    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:19.394194 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:19.394165    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:20.865441 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:20.865407    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff1a5731-c7d8-4a1d-9639-102044250eba" path="/var/lib/kubelet/pods/ff1a5731-c7d8-4a1d-9639-102044250eba/volumes"
Jul 28 20:31:25.445629 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:25.445596    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:25.545444 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:25.545411    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:25.705645 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:25.705568    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:25.920450 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:25.920418    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:31.929968 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:31.929940    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fnmjf_07192589-3f2b-4d02-aee0-35087fd22657/server/0.log"
Jul 28 20:31:32.039591 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:32.039562    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-zxvjp_00096bf5-8c3d-4c1e-9860-ccdb60b4b795/manager/0.log"
Jul 28 20:31:32.150010 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:32.149989    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9ddd48bbd-7tqxm_4b22cfe6-866f-4639-9caf-18ce24cf5748/copy-manifests/0.log"
Jul 28 20:31:32.157438 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:32.157412    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9ddd48bbd-7tqxm_4b22cfe6-866f-4639-9caf-18ce24cf5748/manager/0.log"
Jul 28 20:31:32.762682 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:32.762656    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6f48d844d4-g6v4m_1c8f71f2-d0fb-433e-a181-aa55887b303f/maas-api/0.log"
Jul 28 20:31:32.990272 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:32.990248    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754495-n4xv2_db942e1a-4c20-48c4-b5d3-db5227cf3a98/cleanup/0.log"
Jul 28 20:31:33.962260 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:33.962231    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4_4376f19b-d5bb-44f5-8cdc-eccea2adda49/util/0.log"
Jul 28 20:31:33.968413 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:33.968392    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4_4376f19b-d5bb-44f5-8cdc-eccea2adda49/pull/0.log"
Jul 28 20:31:33.974290 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:33.974269    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4_4376f19b-d5bb-44f5-8cdc-eccea2adda49/extract/0.log"
Jul 28 20:31:34.090922 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:34.090900    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86_cf19ca5b-3313-459c-9dba-08a23169d6f9/extract/0.log"
Jul 28 20:31:34.097730 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:34.097714    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86_cf19ca5b-3313-459c-9dba-08a23169d6f9/util/0.log"
Jul 28 20:31:34.104944 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:34.104927    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86_cf19ca5b-3313-459c-9dba-08a23169d6f9/pull/0.log"
Jul 28 20:31:34.220561 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:34.220499    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9_6e3f64b5-e230-43af-946e-bec7e18d05e7/util/0.log"
Jul 28 20:31:34.228720 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:34.228699    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9_6e3f64b5-e230-43af-946e-bec7e18d05e7/pull/0.log"
Jul 28 20:31:34.236356 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:34.236318    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9_6e3f64b5-e230-43af-946e-bec7e18d05e7/extract/0.log"
Jul 28 20:31:34.349868 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:34.349845    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r_77a8578c-ce3c-4fc7-9a4b-39382ad06958/util/0.log"
Jul 28 20:31:34.359009 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:34.358985    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r_77a8578c-ce3c-4fc7-9a4b-39382ad06958/pull/0.log"
Jul 28 20:31:34.370646 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:34.370624    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r_77a8578c-ce3c-4fc7-9a4b-39382ad06958/extract/0.log"
Jul 28 20:31:35.644654 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:35.644574    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz_ef85412a-fbbd-4ff4-8e42-21bc4e30e444/istio-proxy/0.log"
Jul 28 20:31:35.868305 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:35.868271    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-ccfbc6b9-fw9kp_a29780d1-d9e7-4b75-8b27-a34d95aacec2/kube-auth-proxy/0.log"
Jul 28 20:31:36.097774 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:36.097745    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-c5xr9_36b15eb7-edfa-4803-8270-332cafadae6c/istio-proxy/1.log"
Jul 28 20:31:36.208608 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:36.208582    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-gqmd8_5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210/payload-pre-processing/0.log"
Jul 28 20:31:36.858313 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:36.858260    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-4vdk4_96c06d46-1e9d-46ce-a698-d8d7bda9cb57/main/0.log"
Jul 28 20:31:37.190450 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:37.190361    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-tnxs2_3c069a2c-ac9e-4ec9-b2d4-5b43ed400b91/main/0.log"
Jul 28 20:31:37.295028 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:37.295005    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-m8vpx_81e1b755-4c47-45fd-bf53-d8550103c3b8/main/0.log"
Jul 28 20:31:43.027645 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.027613    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2456d/must-gather-kk4js"]
Jul 28 20:31:43.028024 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.027967    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff1a5731-c7d8-4a1d-9639-102044250eba" containerName="revoke-keys"
Jul 28 20:31:43.028024 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.027977    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff1a5731-c7d8-4a1d-9639-102044250eba" containerName="revoke-keys"
Jul 28 20:31:43.028133 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.028054    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff1a5731-c7d8-4a1d-9639-102044250eba" containerName="revoke-keys"
Jul 28 20:31:43.032666 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.032647    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2456d/must-gather-kk4js"
Jul 28 20:31:43.035358 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.035338    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2456d\"/\"kube-root-ca.crt\""
Jul 28 20:31:43.036263 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.036244    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2456d\"/\"openshift-service-ca.crt\""
Jul 28 20:31:43.036369 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.036250    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2456d\"/\"default-dockercfg-wvrm8\""
Jul 28 20:31:43.042206 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.042183    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2456d/must-gather-kk4js"]
Jul 28 20:31:43.056481 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.056459    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qknvz\" (UniqueName: \"kubernetes.io/projected/16a331b9-4983-4451-afe8-4250553fc97e-kube-api-access-qknvz\") pod \"must-gather-kk4js\" (UID: \"16a331b9-4983-4451-afe8-4250553fc97e\") " pod="openshift-must-gather-2456d/must-gather-kk4js"
Jul 28 20:31:43.056585 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.056520    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/16a331b9-4983-4451-afe8-4250553fc97e-must-gather-output\") pod \"must-gather-kk4js\" (UID: \"16a331b9-4983-4451-afe8-4250553fc97e\") " pod="openshift-must-gather-2456d/must-gather-kk4js"
Jul 28 20:31:43.111010 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.110983    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:43.157381 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.157354    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/16a331b9-4983-4451-afe8-4250553fc97e-must-gather-output\") pod \"must-gather-kk4js\" (UID: \"16a331b9-4983-4451-afe8-4250553fc97e\") " pod="openshift-must-gather-2456d/must-gather-kk4js"
Jul 28 20:31:43.157460 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.157433    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qknvz\" (UniqueName: \"kubernetes.io/projected/16a331b9-4983-4451-afe8-4250553fc97e-kube-api-access-qknvz\") pod \"must-gather-kk4js\" (UID: \"16a331b9-4983-4451-afe8-4250553fc97e\") " pod="openshift-must-gather-2456d/must-gather-kk4js"
Jul 28 20:31:43.157636 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.157618    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/16a331b9-4983-4451-afe8-4250553fc97e-must-gather-output\") pod \"must-gather-kk4js\" (UID: \"16a331b9-4983-4451-afe8-4250553fc97e\") " pod="openshift-must-gather-2456d/must-gather-kk4js"
Jul 28 20:31:43.171083 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.171045    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qknvz\" (UniqueName: \"kubernetes.io/projected/16a331b9-4983-4451-afe8-4250553fc97e-kube-api-access-qknvz\") pod \"must-gather-kk4js\" (UID: \"16a331b9-4983-4451-afe8-4250553fc97e\") " pod="openshift-must-gather-2456d/must-gather-kk4js"
Jul 28 20:31:43.211304 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.211278    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:43.343466 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.343443    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2456d/must-gather-kk4js"
Jul 28 20:31:43.477503 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.477471    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2456d/must-gather-kk4js"]
Jul 28 20:31:43.478945 ip-10-0-138-37 kubenswrapper[2572]: W0728 20:31:43.478918    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16a331b9_4983_4451_afe8_4250553fc97e.slice/crio-eed5eb103dd1148c8f814128257a88a6a755cd3dde55ecd5e2614aed9940ce8e WatchSource:0}: Error finding container eed5eb103dd1148c8f814128257a88a6a755cd3dde55ecd5e2614aed9940ce8e: Status 404 returned error can't find the container with id eed5eb103dd1148c8f814128257a88a6a755cd3dde55ecd5e2614aed9940ce8e
Jul 28 20:31:43.481283 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:43.481266    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:31:44.367379 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:44.367344    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2456d/must-gather-kk4js" event={"ID":"16a331b9-4983-4451-afe8-4250553fc97e","Type":"ContainerStarted","Data":"891e7bf1247f3307daf37cc6c64867fcc1ce786373b7522a7296b2be6a51d217"}
Jul 28 20:31:44.367723 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:44.367399    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2456d/must-gather-kk4js" event={"ID":"16a331b9-4983-4451-afe8-4250553fc97e","Type":"ContainerStarted","Data":"eed5eb103dd1148c8f814128257a88a6a755cd3dde55ecd5e2614aed9940ce8e"}
Jul 28 20:31:45.378341 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:45.378284    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2456d/must-gather-kk4js" event={"ID":"16a331b9-4983-4451-afe8-4250553fc97e","Type":"ContainerStarted","Data":"b787d048b5d184de3867a6894ae8604b53998f6bcd2d7638c83b0acb1783cebb"}
Jul 28 20:31:45.400269 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:45.400222    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2456d/must-gather-kk4js" podStartSLOduration=1.639673765 podStartE2EDuration="2.400206937s" totalImagesPullingTime="760.533172ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:31:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:31:45.396532823 +0000 UTC m=+3730.984195770" watchObservedRunningTime="2026-07-28 20:31:45.400206937 +0000 UTC m=+3730.987869882"
Jul 28 20:31:45.905046 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:45.905011    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vlfs6_2c007a53-c536-4083-ab29-644788d631ba/global-pull-secret-syncer/0.log"
Jul 28 20:31:45.966050 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:45.966023    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log"
Jul 28 20:31:46.127221 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:46.127190    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-37.ec2.internal_70c009bc8352f8c83009ce9d3b8a04d2/haproxy/0.log"
Jul 28 20:31:50.859524 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:50.859482    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4_4376f19b-d5bb-44f5-8cdc-eccea2adda49/extract/0.log"
Jul 28 20:31:50.892802 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:50.892771    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4_4376f19b-d5bb-44f5-8cdc-eccea2adda49/util/0.log"
Jul 28 20:31:50.926823 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:50.926787    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595mcr4_4376f19b-d5bb-44f5-8cdc-eccea2adda49/pull/0.log"
Jul 28 20:31:50.959746 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:50.959708    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86_cf19ca5b-3313-459c-9dba-08a23169d6f9/extract/0.log"
Jul 28 20:31:50.986460 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:50.986433    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86_cf19ca5b-3313-459c-9dba-08a23169d6f9/util/0.log"
Jul 28 20:31:51.014841 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:51.014801    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cdp86_cf19ca5b-3313-459c-9dba-08a23169d6f9/pull/0.log"
Jul 28 20:31:51.047457 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:51.047422    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9_6e3f64b5-e230-43af-946e-bec7e18d05e7/extract/0.log"
Jul 28 20:31:51.072873 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:51.072844    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9_6e3f64b5-e230-43af-946e-bec7e18d05e7/util/0.log"
Jul 28 20:31:51.099999 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:51.099935    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736hxs9_6e3f64b5-e230-43af-946e-bec7e18d05e7/pull/0.log"
Jul 28 20:31:51.135444 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:51.135363    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r_77a8578c-ce3c-4fc7-9a4b-39382ad06958/extract/0.log"
Jul 28 20:31:51.171556 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:51.171522    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r_77a8578c-ce3c-4fc7-9a4b-39382ad06958/util/0.log"
Jul 28 20:31:51.205124 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:51.205093    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltd9r_77a8578c-ce3c-4fc7-9a4b-39382ad06958/pull/0.log"
Jul 28 20:31:54.016836 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:54.016803    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-phhgm_d78b21b7-251a-4508-a29b-8fd37b47b31c/cluster-monitoring-operator/0.log"
Jul 28 20:31:54.151608 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:54.151572    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-q26gl_18b6ab88-4bb2-4798-b7f4-5755b6802b53/monitoring-plugin/0.log"
Jul 28 20:31:54.369857 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:54.369826    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tfqrn_756f572c-dfa1-4d6d-bffa-7c75f093ec47/node-exporter/0.log"
Jul 28 20:31:54.395762 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:54.395733    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tfqrn_756f572c-dfa1-4d6d-bffa-7c75f093ec47/kube-rbac-proxy/0.log"
Jul 28 20:31:54.421477 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:54.421443    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tfqrn_756f572c-dfa1-4d6d-bffa-7c75f093ec47/init-textfile/0.log"
Jul 28 20:31:54.826850 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:54.826804    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66fcc6df55-n9zh2_28abda71-7951-4756-8c6a-13cdba055a55/telemeter-client/0.log"
Jul 28 20:31:54.855276 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:54.855240    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66fcc6df55-n9zh2_28abda71-7951-4756-8c6a-13cdba055a55/reload/0.log"
Jul 28 20:31:54.893523 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:54.893485    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66fcc6df55-n9zh2_28abda71-7951-4756-8c6a-13cdba055a55/kube-rbac-proxy/0.log"
Jul 28 20:31:55.501871 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.501822    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"]
Jul 28 20:31:55.510667 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.510642    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:31:55.513819 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.513778    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"]
Jul 28 20:31:55.586466 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.586426    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn5gc\" (UniqueName: \"kubernetes.io/projected/4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2-kube-api-access-vn5gc\") pod \"perf-node-gather-daemonset-vkbg5\" (UID: \"4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2\") " pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:31:55.586745 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.586611    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2-podres\") pod \"perf-node-gather-daemonset-vkbg5\" (UID: \"4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2\") " pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:31:55.586745 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.586664    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2-lib-modules\") pod \"perf-node-gather-daemonset-vkbg5\" (UID: \"4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2\") " pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:31:55.586886 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.586755    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2-sys\") pod \"perf-node-gather-daemonset-vkbg5\" (UID: \"4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2\") " pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:31:55.586886 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.586799    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2-proc\") pod \"perf-node-gather-daemonset-vkbg5\" (UID: \"4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2\") " pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:31:55.688157 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.688117    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2-lib-modules\") pod \"perf-node-gather-daemonset-vkbg5\" (UID: \"4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2\") " pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:31:55.688401 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.688373    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2-lib-modules\") pod \"perf-node-gather-daemonset-vkbg5\" (UID: \"4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2\") " pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:31:55.688546 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.688435    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2-sys\") pod \"perf-node-gather-daemonset-vkbg5\" (UID: \"4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2\") " pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:31:55.688546 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.688477    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2-proc\") pod \"perf-node-gather-daemonset-vkbg5\" (UID: \"4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2\") " pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:31:55.688546 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.688495    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2-sys\") pod \"perf-node-gather-daemonset-vkbg5\" (UID: \"4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2\") " pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:31:55.688546 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.688527    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn5gc\" (UniqueName: \"kubernetes.io/projected/4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2-kube-api-access-vn5gc\") pod \"perf-node-gather-daemonset-vkbg5\" (UID: \"4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2\") " pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:31:55.688765 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.688625    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2-proc\") pod \"perf-node-gather-daemonset-vkbg5\" (UID: \"4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2\") " pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:31:55.688765 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.688634    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2-podres\") pod \"perf-node-gather-daemonset-vkbg5\" (UID: \"4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2\") " pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:31:55.688765 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.688761    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2-podres\") pod \"perf-node-gather-daemonset-vkbg5\" (UID: \"4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2\") " pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:31:55.698655 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.698626    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn5gc\" (UniqueName: \"kubernetes.io/projected/4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2-kube-api-access-vn5gc\") pod \"perf-node-gather-daemonset-vkbg5\" (UID: \"4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2\") " pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:31:55.823087 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:55.823042    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:31:56.039425 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:56.039394    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"]
Jul 28 20:31:56.295514 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:56.295475    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-txhmp_0bf17d43-9e36-4a59-8cf2-8e23f4ed98f5/networking-console-plugin/0.log"
Jul 28 20:31:56.472736 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:56.472649    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5" event={"ID":"4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2","Type":"ContainerStarted","Data":"e5a3bc268c4973d87f1aa24982a9b7bd8fda8de37ffbeb589367b146ca1240fb"}
Jul 28 20:31:56.472736 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:56.472715    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5" event={"ID":"4b7b30a2-676c-4ac1-bff9-64a3f8b0dad2","Type":"ContainerStarted","Data":"92680213cd540c47a01cfb02608b4af42dc583bc1713211e252840fd24a05653"}
Jul 28 20:31:56.472916 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:56.472847    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:31:56.491367 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:56.491320    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5" podStartSLOduration=1.491304816 podStartE2EDuration="1.491304816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:31:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:31:56.488968865 +0000 UTC m=+3742.076631812" watchObservedRunningTime="2026-07-28 20:31:56.491304816 +0000 UTC m=+3742.078967761"
Jul 28 20:31:56.885501 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:56.885474    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/1.log"
Jul 28 20:31:56.891174 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:56.891148    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jb9gx_a03c223a-531b-4dce-a276-f9ef93d45d28/console-operator/2.log"
Jul 28 20:31:57.358599 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:57.358571    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6f8bbf86d5-clnxs_641696ee-422f-4d04-9ffd-d05f196614a4/console/0.log"
Jul 28 20:31:58.216444 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:58.216393    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5hfx5_fdf14d1f-4a97-49ed-a0f4-f28fd90e5482/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:59.124542 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:59.124511    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6wqlb_eee9f5fc-fb26-4766-86a8-2b2b8b0056c5/dns/0.log"
Jul 28 20:31:59.150185 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:59.150159    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6wqlb_eee9f5fc-fb26-4766-86a8-2b2b8b0056c5/kube-rbac-proxy/0.log"
Jul 28 20:31:59.342435 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:59.342403    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-td8fx_4f0fe6e5-14c7-414a-bea1-7b37dba7b3b5/dns-node-resolver/0.log"
Jul 28 20:31:59.973564 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:31:59.973536    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tgq2d_be4dd071-8c56-455a-bb1e-440b5b292f3b/node-ca/0.log"
Jul 28 20:32:00.935528 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:00.935484    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz_ef85412a-fbbd-4ff4-8e42-21bc4e30e444/istio-proxy/0.log"
Jul 28 20:32:01.111678 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:01.111645    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-ccfbc6b9-fw9kp_a29780d1-d9e7-4b75-8b27-a34d95aacec2/kube-auth-proxy/0.log"
Jul 28 20:32:01.171718 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:01.171692    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-c5xr9_36b15eb7-edfa-4803-8270-332cafadae6c/istio-proxy/0.log"
Jul 28 20:32:01.208214 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:01.208147    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-c5xr9_36b15eb7-edfa-4803-8270-332cafadae6c/istio-proxy/1.log"
Jul 28 20:32:01.247873 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:01.247840    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-gqmd8_5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210/payload-pre-processing/0.log"
Jul 28 20:32:02.161360 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:02.161329    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tpngt_a5c7f30b-8bac-4127-9e0a-44c45b92e47b/serve-healthcheck-canary/0.log"
Jul 28 20:32:02.494716 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:02.494635    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2456d/perf-node-gather-daemonset-vkbg5"
Jul 28 20:32:02.661762 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:02.661731    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-q8b99_ba18d3b8-a8fb-4d09-bc33-c5213185a953/insights-operator/0.log"
Jul 28 20:32:02.662972 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:02.662950    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-q8b99_ba18d3b8-a8fb-4d09-bc33-c5213185a953/insights-operator/1.log"
Jul 28 20:32:02.770592 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:02.770522    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7gmsv_c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf/kube-rbac-proxy/0.log"
Jul 28 20:32:02.796526 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:02.796502    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7gmsv_c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf/exporter/0.log"
Jul 28 20:32:02.821260 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:02.821232    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7gmsv_c60fd0f2-b6d3-450b-9a2e-6a2ca8281ccf/extractor/0.log"
Jul 28 20:32:05.560384 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:05.560321    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz_ef85412a-fbbd-4ff4-8e42-21bc4e30e444/istio-proxy/0.log"
Jul 28 20:32:05.777306 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:05.777273    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fnmjf_07192589-3f2b-4d02-aee0-35087fd22657/server/0.log"
Jul 28 20:32:05.798523 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:05.798500    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-ccfbc6b9-fw9kp_a29780d1-d9e7-4b75-8b27-a34d95aacec2/kube-auth-proxy/0.log"
Jul 28 20:32:05.859293 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:05.859223    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-c5xr9_36b15eb7-edfa-4803-8270-332cafadae6c/istio-proxy/0.log"
Jul 28 20:32:05.905565 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:05.905536    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-c5xr9_36b15eb7-edfa-4803-8270-332cafadae6c/istio-proxy/1.log"
Jul 28 20:32:05.908436 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:05.908415    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-zxvjp_00096bf5-8c3d-4c1e-9860-ccdb60b4b795/manager/0.log"
Jul 28 20:32:05.929395 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:05.929371    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-gqmd8_5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210/payload-pre-processing/0.log"
Jul 28 20:32:05.943007 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:05.942989    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9ddd48bbd-7tqxm_4b22cfe6-866f-4639-9caf-18ce24cf5748/manager/0.log"
Jul 28 20:32:05.979614 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:05.979596    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9ddd48bbd-7tqxm_4b22cfe6-866f-4639-9caf-18ce24cf5748/copy-manifests/0.log"
Jul 28 20:32:07.455178 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:07.455149    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kvqrz_ef85412a-fbbd-4ff4-8e42-21bc4e30e444/istio-proxy/0.log"
Jul 28 20:32:07.610155 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:07.610121    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6c85c6bd85-lqtpg_b406707d-fe20-4134-afd9-f93560ede974/manager/0.log"
Jul 28 20:32:07.630491 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:07.630467    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-ccfbc6b9-fw9kp_a29780d1-d9e7-4b75-8b27-a34d95aacec2/kube-auth-proxy/0.log"
Jul 28 20:32:07.667342 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:07.667315    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-c5xr9_36b15eb7-edfa-4803-8270-332cafadae6c/istio-proxy/0.log"
Jul 28 20:32:07.676841 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:07.676818    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-d2f56_824928c8-3d59-49b6-adc0-77d1df5841d0/openshift-lws-operator/0.log"
Jul 28 20:32:07.701175 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:07.701155    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-c5xr9_36b15eb7-edfa-4803-8270-332cafadae6c/istio-proxy/1.log"
Jul 28 20:32:07.729709 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:07.729679    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-gqmd8_5b1421c8-eed0-42d9-ac4a-6bb7a9fbb210/payload-pre-processing/0.log"
Jul 28 20:32:12.381802 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:12.381769    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-tmtk9_c03a93c7-a1b6-40e3-9bb3-93081ef40fa3/migrator/0.log"
Jul 28 20:32:12.405597 ip-10-0-138-37 kubenswrapper[2572]: I0728 20:32:12.405566    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-tmtk9_c03a93c7-a1b6-40e3-9bb3-93081ef40fa3/graceful-termination/0.log"