Sep 10 17:13:14.334656 ip-10-0-138-123 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 17:13:14.748761 ip-10-0-138-123 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 17:13:14.748761 ip-10-0-138-123 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 17:13:14.748761 ip-10-0-138-123 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 17:13:14.748761 ip-10-0-138-123 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 17:13:14.748761 ip-10-0-138-123 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 17:13:14.750836 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.750731    2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 17:13:14.756062 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756041    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:13:14.756062 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756060    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:13:14.756062 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756064    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756067    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756071    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756075    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756078    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756081    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756084    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756088    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756091    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756094    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756096    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756099    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756102    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756112    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756115    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756118    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756121    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756123    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756126    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:13:14.756165 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756129    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756134    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756138    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756141    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756144    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756147    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756150    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756153    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756156    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756158    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756161    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756164    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756168    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756171    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756174    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756177    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756180    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756182    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756185    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:13:14.756677 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756188    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756190    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756193    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756196    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756198    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756203    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756206    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756209    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756211    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756214    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756217    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756219    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756222    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756224    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756228    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756231    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756234    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756238    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756241    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756243    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:13:14.757144 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756246    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756249    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756251    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756254    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756256    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756259    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756261    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756264    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756266    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756281    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756284    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756287    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756290    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756292    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756296    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756299    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756301    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756304    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756307    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756309    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:13:14.757650 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756312    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756315    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756317    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756320    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756322    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756325    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756328    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756331    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756334    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756337    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756341    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756344    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756347    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756350    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756352    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756355    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756357    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756360    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.756362    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757122    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:13:14.758146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757129    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757132    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757135    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757138    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757142    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757145    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757149    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757152    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757155    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757159    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757162    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757165    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757167    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757170    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757173    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757175    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757178    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757180    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757184    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757187    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:13:14.758672 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757190    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757193    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757195    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757198    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757201    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757204    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757206    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757209    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757211    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757214    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757217    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757219    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757221    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757224    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757227    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757229    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757232    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757235    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757237    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757241    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:13:14.759218 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757244    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757247    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757250    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757253    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757255    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757258    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757260    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757263    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757265    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757282    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757285    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757288    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757291    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757294    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757296    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757299    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757301    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757304    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757307    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757309    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:13:14.759728 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757312    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757314    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757317    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757320    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757322    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757325    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757327    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757332    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757336    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757338    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757342    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757345    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757348    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757352    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757355    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757357    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757360    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757363    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757367    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:13:14.760232 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757370    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757372    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757375    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757378    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757380    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757383    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757386    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757388    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757391    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757393    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757396    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757400    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757404    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757407    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757410    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757413    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757416    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757418    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757421    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:13:14.760720 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757843    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757849    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757852    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757855    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757861    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757864    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757867    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757870    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757873    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757876    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757879    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757881    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757884    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757886    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757889    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757891    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757894    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757896    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757899    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757902    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:13:14.761189 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757904    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757907    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757910    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757912    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757914    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757917    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757920    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757922    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757925    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757927    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757930    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757932    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757935    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757937    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757940    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757944    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757947    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757951    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757955    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:13:14.761812 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757959    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757962    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757965    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757967    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757970    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757972    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757975    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757978    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757980    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757983    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757985    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757988    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757991    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757994    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757996    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.757998    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758001    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758003    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758006    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758008    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:13:14.762292 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758011    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758013    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758015    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758018    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758020    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758023    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758026    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758028    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758031    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758033    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758036    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758042    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758044    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758047    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758049    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758052    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758055    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758057    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758060    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:13:14.762784 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758064    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758067    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758070    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758073    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758076    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758078    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758081    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758085    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758087    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758090    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758093    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758096    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758098    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758101    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758103    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758106    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758108    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758111    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758114    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758117    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:13:14.763267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758120    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758204    2566 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758240    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758252    2566 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758259    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758264    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758285    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758290    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758295    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758299    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758302    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758306    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758309    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758312    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758315    2566 flags.go:64] FLAG: --cgroup-root=""
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758318    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758322    2566 flags.go:64] FLAG: --client-ca-file=""
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758325    2566 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758328    2566 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758333    2566 flags.go:64] FLAG: --cluster-domain=""
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758336    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758339    2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758342    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758346    2566 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 17:13:14.763780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758350    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758354    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758357    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758361    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758364    2566 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758367    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758370    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758373    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758376    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758381    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758384    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758388    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758393    2566 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758396    2566 flags.go:64] FLAG: --enable-server="true"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758400    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758404    2566 flags.go:64] FLAG: --event-burst="100"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758407    2566 flags.go:64] FLAG: --event-qps="50"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758411    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758414    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758417    2566 flags.go:64] FLAG: --eviction-hard=""
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758421    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758424    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758427    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758430    2566 flags.go:64] FLAG: --eviction-soft=""
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758433    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 17:13:14.764398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758436    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758439    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758442    2566 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758444    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758448    2566 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758450    2566 flags.go:64] FLAG: --feature-gates=""
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758454    2566 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758457    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758461    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758464    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758467    2566 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758470    2566 flags.go:64] FLAG: --help="false"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758473    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-138-123.ec2.internal"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758476    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758480    2566 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758483    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758486    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758490    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758494    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758498    2566 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758501    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758504    2566 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758508    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758512    2566 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 17:13:14.765064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758515    2566 flags.go:64] FLAG: --kube-reserved=""
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758518    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758521    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758524    2566 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758528    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758531    2566 flags.go:64] FLAG: --lock-file=""
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758534    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758537    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758540    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758546    2566 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758549    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758551    2566 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758554    2566 flags.go:64] FLAG: --logging-format="text"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758557    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758560    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758563    2566 flags.go:64] FLAG: --manifest-url=""
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758566    2566 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758571    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758574    2566 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758578    2566 flags.go:64] FLAG: --max-pods="110"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758581    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758584    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758587    2566 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758590    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758593    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 17:13:14.765695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758596    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758600    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758609    2566 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758612    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758615    2566 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758618    2566 flags.go:64] FLAG: --pod-cidr=""
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758622    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758628    2566 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758631    2566 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758634    2566 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758637    2566 flags.go:64] FLAG: --port="10250"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758640    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758644    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03531f9b322e07066"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758647    2566 flags.go:64] FLAG: --qos-reserved=""
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758650    2566 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758654    2566 flags.go:64] FLAG: --register-node="true"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758657    2566 flags.go:64] FLAG: --register-with-taints=""
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758660    2566 flags.go:64] FLAG: --registry-burst="10"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758664    2566 flags.go:64] FLAG: --registry-qps="5"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758666    2566 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758669    2566 flags.go:64] FLAG: --reserved-memory=""
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758673    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758676    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758679    2566 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758682    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 17:13:14.766634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758685    2566 flags.go:64] FLAG: --runonce="false"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758688    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758692    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758695    2566 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758698    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758701    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758704    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758707    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758710    2566 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758716    2566 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758719    2566 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758721    2566 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758724    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758727    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758731    2566 flags.go:64] FLAG: --system-cgroups=""
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758734    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758739    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758742    2566 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758745    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758749    2566 flags.go:64] FLAG: --tls-min-version=""
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758752    2566 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758755    2566 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758758    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758761    2566 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758763    2566 flags.go:64] FLAG: --v="2"
Sep 10 17:13:14.767717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758768    2566 flags.go:64] FLAG: --version="false"
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758772    2566 flags.go:64] FLAG: --vmodule=""
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758776    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.758779    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758900    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758904    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758907    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758910    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758913    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758916    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758920    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758923    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758926    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758928    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758931    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758934    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758941    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758944    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758947    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758950    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:13:14.768641 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758953    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758956    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758960    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758963    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758966    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758969    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758971    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758974    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758977    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758980    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758982    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758984    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758987    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758989    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758992    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758995    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.758997    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759000    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759004    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759008    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:13:14.769184 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759011    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759014    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759016    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759019    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759022    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759026    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759029    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759032    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759039    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759041    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759044    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759047    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759050    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759052    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759055    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759058    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759061    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759064    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759066    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:13:14.770028 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759069    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759071    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759074    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759077    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759080    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759084    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759088    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759090    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759093    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759096    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759098    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759101    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759103    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759106    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759109    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759111    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759114    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759116    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759119    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759121    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:13:14.770846 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759124    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759130    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759132    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759135    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759138    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759141    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759144    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759146    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759149    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759151    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759154    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759156    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759159    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759162    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759165    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759168    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759170    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759173    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759176    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759179    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:13:14.771467 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759181    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:13:14.772106 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759184    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:13:14.772106 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759186    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:13:14.772106 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.759189    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:13:14.772106 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.759195    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 17:13:14.772106 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.768320    2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 17:13:14.772106 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.768343    2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 17:13:14.772106 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.769471    2566 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 17:13:14.772106 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.769492    2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 17:13:14.772106 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769596    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:13:14.772106 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769604    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:13:14.772106 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769609    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:13:14.772106 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769613    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:13:14.772106 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769618    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:13:14.772106 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769623    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:13:14.772106 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769628    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769633    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769637    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769641    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769646    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769650    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769656    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769668    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769672    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769677    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769681    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769688    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769694    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769699    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769703    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769709    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769713    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769718    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769722    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:13:14.772763 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769726    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769731    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769735    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769739    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769745    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769749    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769754    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769758    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769763    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769767    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769771    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769775    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769779    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769784    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769788    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769793    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769797    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769801    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769805    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769809    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:13:14.773539 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769814    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769818    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769823    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769826    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769830    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769835    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769839    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769844    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769848    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769852    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769858    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769863    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769869    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769874    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769878    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769883    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769888    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769893    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769897    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:13:14.774380 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769901    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769906    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769910    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769915    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769919    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769923    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769927    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769932    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769936    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769940    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769944    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769948    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769952    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769957    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769962    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769966    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769971    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769975    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769979    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769983    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:13:14.774866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769987    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:13:14.775366 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769994    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:13:14.775366 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.769999    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:13:14.775366 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770003    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:13:14.775366 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770007    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:13:14.775366 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770011    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:13:14.775366 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770016    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:13:14.775366 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770020    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:13:14.775366 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770024    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:13:14.775366 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770029    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:13:14.775366 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770033    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:13:14.775366 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770037    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:13:14.775366 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770041    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:13:14.775366 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770045    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:13:14.775366 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770049    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:13:14.775719 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.770057    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 17:13:14.775719 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770251    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:13:14.775719 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770260    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:13:14.775719 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770265    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:13:14.775719 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770286    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:13:14.775719 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770291    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:13:14.775719 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770295    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:13:14.775719 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770299    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:13:14.775719 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770303    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:13:14.775719 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770308    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:13:14.775719 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770312    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:13:14.775719 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770316    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:13:14.775719 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770322    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:13:14.775719 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770326    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:13:14.775719 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770330    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770335    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770339    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770343    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770348    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770352    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770356    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770360    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770364    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770368    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770372    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770377    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770382    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770387    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770391    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770396    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770401    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770406    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770410    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770414    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:13:14.776094 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770418    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770423    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770427    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770432    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770437    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770441    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770445    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770450    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770454    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770457    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770462    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770467    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770471    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770475    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770480    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770484    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770488    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770493    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770497    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770501    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:13:14.776611 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770506    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770511    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770517    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770523    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770528    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770533    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770538    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770542    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770547    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770552    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770557    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770561    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770566    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770570    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770574    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770578    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770582    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770586    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770590    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:13:14.777098 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770596    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770602    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770606    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770611    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770616    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770621    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770625    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770630    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770635    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770639    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770643    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770647    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770652    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770656    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770661    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770665    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770670    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770676    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770681    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770685    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:13:14.777585 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770689    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:13:14.778077 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770693    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:13:14.778077 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770697    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:13:14.778077 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770701    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:13:14.778077 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770705    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:13:14.778077 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770709    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:13:14.778077 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:14.770713    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:13:14.778077 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.770722    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 17:13:14.778077 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.770916    2566 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 17:13:14.778077 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.774428    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 17:13:14.778077 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.775874    2566 server.go:1019] "Starting client certificate rotation"
Sep 10 17:13:14.778077 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.775980    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 17:13:14.778077 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.776017    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 17:13:14.778849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.778832    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 17:13:14.780117 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.780098    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 17:13:14.790157 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.790133    2566 log.go:25] "Validated CRI v1 runtime API"
Sep 10 17:13:14.793640 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.793623    2566 log.go:25] "Validated CRI v1 image API"
Sep 10 17:13:14.796842 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.796825    2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 17:13:14.799480 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.799455    2566 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7bd1627e-a033-424d-888e-b75bc1acd60b:/dev/nvme0n1p3 87325a60-2c92-444d-8a99-2fd7d06f5cbf:/dev/nvme0n1p4]
Sep 10 17:13:14.799533 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.799479    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 17:13:14.804241 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.804119    2566 manager.go:216] Machine: {Timestamp:2026-09-10 17:13:14.803609907 +0000 UTC m=+0.367635595 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3073264 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25371cd43c1b3eae6533982c5f37b3 SystemUUID:ec25371c-d43c-1b3e-ae65-33982c5f37b3 BootID:03f4d43f-3cd9-4916-85ab-f480e4fd067f Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6f:5d:4d:f2:eb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6f:5d:4d:f2:eb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:13:7d:f9:52:d8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 17:13:14.804241 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.804235    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 17:13:14.804361 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.804345    2566 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 17:13:14.807519 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.807501    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 17:13:14.808100 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.808067    2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 17:13:14.808253 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.808104    2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-123.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 17:13:14.809002 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.808989    2566 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 17:13:14.809038 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.809004    2566 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 17:13:14.809038 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.809013    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 17:13:14.809989 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.809975    2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 17:13:14.811853 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.811842    2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 17:13:14.818147 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.818126    2566 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 17:13:14.818308 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.818296    2566 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 17:13:14.818353 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.818338    2566 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 17:13:14.818381 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.818352    2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 17:13:14.818381 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.818369    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 17:13:14.818381 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.818381    2566 kubelet.go:398] "Adding apiserver pod source"
Sep 10 17:13:14.818454 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.818391    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 17:13:14.818817 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.818803    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 17:13:14.818878 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.818823    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 17:13:14.822950 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.822918    2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 17:13:14.825999 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.825984    2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 17:13:14.826081 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.826016    2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 17:13:14.826201 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.826191    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 17:13:14.826236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.826207    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 17:13:14.826236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.826214    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 17:13:14.826236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.826222    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 17:13:14.826236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.826233    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 17:13:14.826360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.826242    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 17:13:14.826816 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.826807    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 17:13:14.826816 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.826816    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 17:13:14.826882 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.826824    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 17:13:14.826882 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.826830    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 17:13:14.826882 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.826838    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 17:13:14.826882 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.826846    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 17:13:14.827727 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.827716    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 17:13:14.827727 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.827726    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 17:13:14.828026 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:14.828006    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-123.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 17:13:14.828064 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:14.828007    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 17:13:14.831941 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.831925    2566 server.go:1268] "Started kubelet"
Sep 10 17:13:14.832022 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.831994    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 17:13:14.832140 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.832078    2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 17:13:14.832204 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.832172    2566 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 17:13:14.832351 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.832334    2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-123.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 17:13:14.832866 ip-10-0-138-123 systemd[1]: Started Kubernetes Kubelet.
Sep 10 17:13:14.834040 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.834022    2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 17:13:14.834204 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.834160    2566 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 17:13:14.837511 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:14.837492    2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 17:13:14.840672 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.840649    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 17:13:14.841389 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.841369    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 17:13:14.841528 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:14.841509    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-123.ec2.internal\" not found"
Sep 10 17:13:14.841869 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:14.840612    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-123.ec2.internal.18d404f343acc331  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-123.ec2.internal,UID:ip-10-0-138-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-123.ec2.internal,},FirstTimestamp:2026-09-10 17:13:14.831889201 +0000 UTC m=+0.395914887,LastTimestamp:2026-09-10 17:13:14.831889201 +0000 UTC m=+0.395914887,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-123.ec2.internal,}"
Sep 10 17:13:14.842243 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.842226    2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 17:13:14.842243 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.842230    2566 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 17:13:14.842384 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.842261    2566 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 17:13:14.842445 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.842432    2566 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 17:13:14.842445 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.842444    2566 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 17:13:14.843185 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.843167    2566 factory.go:153] Registering CRI-O factory
Sep 10 17:13:14.843253 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.843204    2566 factory.go:223] Registration of the crio container factory successfully
Sep 10 17:13:14.843301 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.843256    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 17:13:14.843301 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.843266    2566 factory.go:55] Registering systemd factory
Sep 10 17:13:14.843301 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.843288    2566 factory.go:223] Registration of the systemd container factory successfully
Sep 10 17:13:14.843410 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.843306    2566 factory.go:103] Registering Raw factory
Sep 10 17:13:14.843410 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.843320    2566 manager.go:1196] Started watching for new ooms in manager
Sep 10 17:13:14.843726 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.843712    2566 manager.go:349] Starting recovery of all containers
Sep 10 17:13:14.844174 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:14.844146    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 17:13:14.844251 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:14.844183    2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-123.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 17:13:14.854802 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.854772    2566 manager.go:354] Recovery completed
Sep 10 17:13:14.860329 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.860162    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:13:14.862498 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.862476    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:13:14.862602 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.862516    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:13:14.862602 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.862532    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:13:14.863205 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.863188    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 17:13:14.863205 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.863197    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:13:14.863205 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.863203    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 17:13:14.863357 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.863219    2566 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 17:13:14.863492 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.863476    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nh9tw"
Sep 10 17:13:14.864229 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.864211    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:13:14.864317 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.864242    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:13:14.864317 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.864253    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:13:14.865425 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.865411    2566 policy_none.go:49] "None policy: Start"
Sep 10 17:13:14.865425 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.865427    2566 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 17:13:14.865516 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.865437    2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 17:13:14.867008 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:14.864884    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-123.ec2.internal.18d404f3457fd076  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-123.ec2.internal,UID:ip-10-0-138-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-123.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-123.ec2.internal,},FirstTimestamp:2026-09-10 17:13:14.86249791 +0000 UTC m=+0.426523599,LastTimestamp:2026-09-10 17:13:14.86249791 +0000 UTC m=+0.426523599,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-123.ec2.internal,}"
Sep 10 17:13:14.867409 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.867393    2566 policy_none.go:47] "Start"
Sep 10 17:13:14.871074 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.871054    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nh9tw"
Sep 10 17:13:14.875261 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:14.875188    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-123.ec2.internal.18d404f3458034ca  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-123.ec2.internal,UID:ip-10-0-138-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-123.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-123.ec2.internal,},FirstTimestamp:2026-09-10 17:13:14.862523594 +0000 UTC m=+0.426549280,LastTimestamp:2026-09-10 17:13:14.862523594 +0000 UTC m=+0.426549280,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-123.ec2.internal,}"
Sep 10 17:13:14.906018 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.905994    2566 manager.go:337] "Starting Device Plugin manager"
Sep 10 17:13:14.924750 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:14.906045    2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 17:13:14.924750 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.906060    2566 server.go:87] "Starting device plugin registration server"
Sep 10 17:13:14.924750 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.906530    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 17:13:14.924750 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.906544    2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 17:13:14.924750 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.906666    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 17:13:14.924750 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.906842    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 17:13:14.924750 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.906852    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 17:13:14.924750 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:14.907442    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 17:13:14.924750 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:14.907493    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-123.ec2.internal\" not found"
Sep 10 17:13:14.942991 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.942945    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 17:13:14.944327 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.944311    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 17:13:14.944394 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.944366    2566 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 17:13:14.944394 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.944392    2566 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 17:13:14.944539 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:14.944505    2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 17:13:14.946333 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:14.946311    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:13:15.007444 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.007330    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:13:15.008354 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.008334    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:13:15.008472 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.008366    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:13:15.008472 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.008377    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:13:15.008472 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.008403    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-123.ec2.internal"
Sep 10 17:13:15.016410 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.016391    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-123.ec2.internal"
Sep 10 17:13:15.016479 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:15.016416    2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-123.ec2.internal\": node \"ip-10-0-138-123.ec2.internal\" not found"
Sep 10 17:13:15.045351 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.045292    2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-123.ec2.internal"]
Sep 10 17:13:15.045465 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.045415    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:13:15.046470 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.046452    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:13:15.046557 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.046483    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:13:15.046557 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.046493    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:13:15.048805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.048792    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:13:15.048954 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.048939    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal"
Sep 10 17:13:15.048999 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.048979    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:13:15.049680 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.049654    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:13:15.049792 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.049693    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:13:15.049792 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.049654    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:13:15.049792 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.049711    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:13:15.049792 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.049723    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:13:15.049792 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.049737    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:13:15.049973 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:15.049833    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-123.ec2.internal\" not found"
Sep 10 17:13:15.052137 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.052118    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-123.ec2.internal"
Sep 10 17:13:15.052189 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.052153    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:13:15.052908 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.052893    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:13:15.052976 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.052925    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:13:15.052976 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.052952    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:13:15.078258 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:15.078224    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-123.ec2.internal\" not found" node="ip-10-0-138-123.ec2.internal"
Sep 10 17:13:15.082812 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:15.082792    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-123.ec2.internal\" not found" node="ip-10-0-138-123.ec2.internal"
Sep 10 17:13:15.150409 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:15.150363    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-123.ec2.internal\" not found"
Sep 10 17:13:15.244325 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.244266    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cd25bbcec769cf1adcfbe899ed035566-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal\" (UID: \"cd25bbcec769cf1adcfbe899ed035566\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal"
Sep 10 17:13:15.244416 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.244328    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd25bbcec769cf1adcfbe899ed035566-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal\" (UID: \"cd25bbcec769cf1adcfbe899ed035566\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal"
Sep 10 17:13:15.244416 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.244349    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7691c42ef3bfce427afbddd33d4f4b3-config\") pod \"kube-apiserver-proxy-ip-10-0-138-123.ec2.internal\" (UID: \"b7691c42ef3bfce427afbddd33d4f4b3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-123.ec2.internal"
Sep 10 17:13:15.251389 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:15.251351    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-123.ec2.internal\" not found"
Sep 10 17:13:15.345216 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.345143    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cd25bbcec769cf1adcfbe899ed035566-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal\" (UID: \"cd25bbcec769cf1adcfbe899ed035566\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal"
Sep 10 17:13:15.345216 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.345183    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd25bbcec769cf1adcfbe899ed035566-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal\" (UID: \"cd25bbcec769cf1adcfbe899ed035566\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal"
Sep 10 17:13:15.345216 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.345200    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7691c42ef3bfce427afbddd33d4f4b3-config\") pod \"kube-apiserver-proxy-ip-10-0-138-123.ec2.internal\" (UID: \"b7691c42ef3bfce427afbddd33d4f4b3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-123.ec2.internal"
Sep 10 17:13:15.345374 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.345246    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7691c42ef3bfce427afbddd33d4f4b3-config\") pod \"kube-apiserver-proxy-ip-10-0-138-123.ec2.internal\" (UID: \"b7691c42ef3bfce427afbddd33d4f4b3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-123.ec2.internal"
Sep 10 17:13:15.345374 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.345241    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cd25bbcec769cf1adcfbe899ed035566-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal\" (UID: \"cd25bbcec769cf1adcfbe899ed035566\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal"
Sep 10 17:13:15.345374 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:15.345287    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd25bbcec769cf1adcfbe899ed035566-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal\" (UID: \"cd25bbcec769cf1adcfbe899ed035566\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal"
Sep 10 17:13:15.352254 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:15.352233    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-123.ec2.internal\" not found"
Sep 10 17:13:17.264308 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:17.264274    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal"
Sep 10 17:13:17.268376 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:17.268353    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-123.ec2.internal"
Sep 10 17:13:17.335548 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:17.335498    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-123.ec2.internal\" not found"
Sep 10 17:13:17.436178 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:17.436142    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-123.ec2.internal\" not found"
Sep 10 17:13:17.536657 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:17.536619    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-123.ec2.internal\" not found"
Sep 10 17:13:17.637307 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:17.637267    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-123.ec2.internal\" not found"
Sep 10 17:13:17.659430 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:17.659400    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 17:13:17.659600 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:17.659581    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 17:13:17.738223 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:17.738191    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-123.ec2.internal\" not found"
Sep 10 17:13:17.742359 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:17.742334    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:13:17.758705 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:17.758628    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 17:08:16 +0000 UTC" deadline="2028-03-03 11:59:00.032018103 +0000 UTC"
Sep 10 17:13:17.758705 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:17.758669    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12954h45m42.273352937s"
Sep 10 17:13:17.806813 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:17.806739    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:13:17.824975 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:17.824945    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal"
Sep 10 17:13:17.833207 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:17.833176    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7691c42ef3bfce427afbddd33d4f4b3.slice/crio-922a23fe5b62c0c7803e4c944f434c1c8b2e76f477d6a4ab8d7547197593c0a7 WatchSource:0}: Error finding container 922a23fe5b62c0c7803e4c944f434c1c8b2e76f477d6a4ab8d7547197593c0a7: Status 404 returned error can't find the container with id 922a23fe5b62c0c7803e4c944f434c1c8b2e76f477d6a4ab8d7547197593c0a7
Sep 10 17:13:17.838748 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:17.838724    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 17:13:17.839567 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:17.839545    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 17:13:17.840023 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:17.840003    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-123.ec2.internal"
Sep 10 17:13:17.844394 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:17.844370    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd25bbcec769cf1adcfbe899ed035566.slice/crio-cbc531582766d3bf3b2a1b16476553f3859e66e8d70927137cce9bcc18dc0c53 WatchSource:0}: Error finding container cbc531582766d3bf3b2a1b16476553f3859e66e8d70927137cce9bcc18dc0c53: Status 404 returned error can't find the container with id cbc531582766d3bf3b2a1b16476553f3859e66e8d70927137cce9bcc18dc0c53
Sep 10 17:13:17.848601 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:17.848585    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 17:13:17.893218 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:17.893183    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:13:18.702787 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.702748    2566 apiserver.go:52] "Watching apiserver"
Sep 10 17:13:18.710474 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.709991    2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 17:13:18.712434 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.712401    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-qxx2h","kube-system/global-pull-secret-syncer-qqw5f","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn","openshift-multus/multus-additional-cni-plugins-tp7gl","openshift-multus/multus-gmg9v","openshift-network-diagnostics/network-check-target-l4rtn","openshift-ovn-kubernetes/ovnkube-node-ssbcl","kube-system/konnectivity-agent-mn5ts","kube-system/kube-apiserver-proxy-ip-10-0-138-123.ec2.internal","openshift-cluster-node-tuning-operator/tuned-nsv5g","openshift-dns/node-resolver-fl4lm","openshift-image-registry/node-ca-plvzg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal","openshift-multus/network-metrics-daemon-rzqqc"]
Sep 10 17:13:18.715253 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.715088    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.717747 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.717722    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:18.717893 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:18.717826    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzqqc" podUID="7a5b4316-2391-4e12-9a80-12ace78ee49d"
Sep 10 17:13:18.718700 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.718677    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 17:13:18.718879 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.718855    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 17:13:18.718943 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.718899    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 17:13:18.719182 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.719012    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 17:13:18.719547 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.719391    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-65ggg\""
Sep 10 17:13:18.720330 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.719825    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-plvzg"
Sep 10 17:13:18.722969 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.722387    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 17:13:18.722969 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.722678    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 17:13:18.722969 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.722681    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 17:13:18.722969 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.722733    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bq56m\""
Sep 10 17:13:18.723198 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.722996    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.726710 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.724642    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 17:13:18.726710 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.726006    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.726710 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.726412    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-46qgl\""
Sep 10 17:13:18.726710 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.726646    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 17:13:18.727484 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.726741    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 17:13:18.728253 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.727962    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:18.728253 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:18.728026    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4rtn" podUID="d993b686-b64a-4643-b53e-2021d0e476ee"
Sep 10 17:13:18.730304 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.730282    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 17:13:18.730533 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.730520    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 17:13:18.730880 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.730781    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qxx2h"
Sep 10 17:13:18.730880 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.730843    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 17:13:18.730880 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.730866    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 17:13:18.731049 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.730902    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 17:13:18.731243 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.731213    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2vj55\""
Sep 10 17:13:18.731243 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.731229    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 17:13:18.733150 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.732960    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mn5ts"
Sep 10 17:13:18.733460 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.733443    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 17:13:18.733774 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.733670    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 17:13:18.733879 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.733845    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 17:13:18.734456 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.734216    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6kn7s\""
Sep 10 17:13:18.736623 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.736599    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 17:13:18.736793 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.736767    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 17:13:18.737074 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737057    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4cdcd\""
Sep 10 17:13:18.737192 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737168    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-host-var-lib-cni-multus\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.737256 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737216    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-multus-conf-dir\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.737310 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737254    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs\") pod \"network-metrics-daemon-rzqqc\" (UID: \"7a5b4316-2391-4e12-9a80-12ace78ee49d\") " pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:18.737310 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737285    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9efe55c6-4eef-4472-846d-4c3861daefa1-cnibin\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.737411 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737397    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhtxp\" (UniqueName: \"kubernetes.io/projected/9efe55c6-4eef-4472-846d-4c3861daefa1-kube-api-access-nhtxp\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.737466 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737438    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-run-systemd\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.737516 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737503    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-var-lib-openvswitch\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.737567 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737540    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7023d897-e2aa-40ea-be34-840f36db1f8d-ovn-node-metrics-cert\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.737618 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737586    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-multus-cni-dir\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.737668 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737612    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-cni-netd\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.737668 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737643    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfmn6\" (UniqueName: \"kubernetes.io/projected/26f57ed5-a830-4098-a2ae-651e7cd00567-kube-api-access-sfmn6\") pod \"node-ca-plvzg\" (UID: \"26f57ed5-a830-4098-a2ae-651e7cd00567\") " pod="openshift-image-registry/node-ca-plvzg"
Sep 10 17:13:18.737768 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737678    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9efe55c6-4eef-4472-846d-4c3861daefa1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.737768 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737717    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-slash\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.737863 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737770    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7023d897-e2aa-40ea-be34-840f36db1f8d-ovnkube-config\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.737863 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737809    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2fa83261-0f33-4437-b34b-ccb301ff0ac9-cni-binary-copy\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.737863 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737853    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-host-var-lib-kubelet\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.738013 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737895    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-hostroot\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.738013 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737950    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-kubelet\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.738013 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.737989    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-systemd-units\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.738165 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738047    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-node-log\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.738165 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738103    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plszb\" (UniqueName: \"kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb\") pod \"network-check-target-l4rtn\" (UID: \"d993b686-b64a-4643-b53e-2021d0e476ee\") " pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:18.738165 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738151    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/26f57ed5-a830-4098-a2ae-651e7cd00567-serviceca\") pod \"node-ca-plvzg\" (UID: \"26f57ed5-a830-4098-a2ae-651e7cd00567\") " pod="openshift-image-registry/node-ca-plvzg"
Sep 10 17:13:18.738318 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738186    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9efe55c6-4eef-4472-846d-4c3861daefa1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.738318 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738248    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-cni-bin\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.738318 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738280    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-cnibin\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.738318 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738312    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbg2m\" (UniqueName: \"kubernetes.io/projected/7a5b4316-2391-4e12-9a80-12ace78ee49d-kube-api-access-bbg2m\") pod \"network-metrics-daemon-rzqqc\" (UID: \"7a5b4316-2391-4e12-9a80-12ace78ee49d\") " pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:18.738501 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738342    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-run-netns\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.738501 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738371    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-run-ovn\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.738501 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738402    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-log-socket\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.738501 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738475    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-multus-socket-dir-parent\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.738695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738521    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-host-run-k8s-cni-cncf-io\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.738695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738560    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2fa83261-0f33-4437-b34b-ccb301ff0ac9-multus-daemon-config\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.738695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738597    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-host-run-multus-certs\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.738695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738648    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-etc-kubernetes\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.738879 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738694    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9efe55c6-4eef-4472-846d-4c3861daefa1-system-cni-dir\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.738879 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738727    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9efe55c6-4eef-4472-846d-4c3861daefa1-cni-binary-copy\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.738879 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738767    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-etc-openvswitch\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.738879 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738808    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-run-openvswitch\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.738879 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738848    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-run-ovn-kubernetes\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.738879 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738875    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c56k\" (UniqueName: \"kubernetes.io/projected/2fa83261-0f33-4437-b34b-ccb301ff0ac9-kube-api-access-4c56k\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.739160 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738918    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26f57ed5-a830-4098-a2ae-651e7cd00567-host\") pod \"node-ca-plvzg\" (UID: \"26f57ed5-a830-4098-a2ae-651e7cd00567\") " pod="openshift-image-registry/node-ca-plvzg"
Sep 10 17:13:18.739160 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738947    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9efe55c6-4eef-4472-846d-4c3861daefa1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.739160 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.738985    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7023d897-e2aa-40ea-be34-840f36db1f8d-env-overrides\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.739160 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.739015    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7023d897-e2aa-40ea-be34-840f36db1f8d-ovnkube-script-lib\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.739160 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.739044    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbsnv\" (UniqueName: \"kubernetes.io/projected/7023d897-e2aa-40ea-be34-840f36db1f8d-kube-api-access-pbsnv\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.739160 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.739075    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-system-cni-dir\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.739421 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.739160    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.739421 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.739212    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-host-run-netns\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.739421 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.739240    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-host-var-lib-cni-bin\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.739421 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.739331    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 17:13:18.741190 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.741116    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.743882 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.743569    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 17:13:18.743882 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.743579    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pqndg\""
Sep 10 17:13:18.743882 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.743709    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 17:13:18.743882 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.743793    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fl4lm"
Sep 10 17:13:18.746704 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.746660    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.749186 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.748927    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:18.749186 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:18.748996    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqw5f" podUID="5d278a9b-4d67-4eaf-a7a4-d2e980c2a775"
Sep 10 17:13:18.749425 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.749408    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-z8w2l\""
Sep 10 17:13:18.749733 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.749706    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 17:13:18.749891 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.749873    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 17:13:18.750371 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.750327    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 17:13:18.750944 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.750927    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 17:13:18.751109 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.751075    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 17:13:18.751357 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.751341    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mkhpt\""
Sep 10 17:13:18.761232 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.761084    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-chzj7"
Sep 10 17:13:18.772953 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.772931    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-chzj7"
Sep 10 17:13:18.776880 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.776683    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:13:18.826934 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.826902    2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 17:13:18.833606 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.833510    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-123.ec2.internal" event={"ID":"b7691c42ef3bfce427afbddd33d4f4b3","Type":"ContainerStarted","Data":"922a23fe5b62c0c7803e4c944f434c1c8b2e76f477d6a4ab8d7547197593c0a7"}
Sep 10 17:13:18.835101 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.835072    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal" event={"ID":"cd25bbcec769cf1adcfbe899ed035566","Type":"ContainerStarted","Data":"cbc531582766d3bf3b2a1b16476553f3859e66e8d70927137cce9bcc18dc0c53"}
Sep 10 17:13:18.840138 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840091    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-tuned\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.840291 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840149    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/85265614-4740-49c7-aa44-934ee4ea6fde-sys-fs\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.840291 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840204    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7023d897-e2aa-40ea-be34-840f36db1f8d-ovnkube-script-lib\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.840291 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840240    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-host-var-lib-cni-bin\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.840291 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840268    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzfk7\" (UniqueName: \"kubernetes.io/projected/69cd67eb-da77-4826-9bc2-a5202871c135-kube-api-access-nzfk7\") pod \"node-resolver-fl4lm\" (UID: \"69cd67eb-da77-4826-9bc2-a5202871c135\") " pod="openshift-dns/node-resolver-fl4lm"
Sep 10 17:13:18.840494 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840291    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb-host-slash\") pod \"iptables-alerter-qxx2h\" (UID: \"0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb\") " pod="openshift-network-operator/iptables-alerter-qxx2h"
Sep 10 17:13:18.840494 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840317    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-dbus\") pod \"global-pull-secret-syncer-qqw5f\" (UID: \"5d278a9b-4d67-4eaf-a7a4-d2e980c2a775\") " pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:18.840494 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840341    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9efe55c6-4eef-4472-846d-4c3861daefa1-cnibin\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.840494 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840366    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhtxp\" (UniqueName: \"kubernetes.io/projected/9efe55c6-4eef-4472-846d-4c3861daefa1-kube-api-access-nhtxp\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.840494 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840389    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/85265614-4740-49c7-aa44-934ee4ea6fde-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.840494 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840412    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/85265614-4740-49c7-aa44-934ee4ea6fde-device-dir\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.840494 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840437    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7023d897-e2aa-40ea-be34-840f36db1f8d-ovnkube-config\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.840494 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840464    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2fa83261-0f33-4437-b34b-ccb301ff0ac9-cni-binary-copy\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.840494 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840488    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-hostroot\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.840947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840533    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/69cd67eb-da77-4826-9bc2-a5202871c135-hosts-file\") pod \"node-resolver-fl4lm\" (UID: \"69cd67eb-da77-4826-9bc2-a5202871c135\") " pod="openshift-dns/node-resolver-fl4lm"
Sep 10 17:13:18.840947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840570    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-kubelet\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.840947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840593    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-sysconfig\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.840947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840619    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/26f57ed5-a830-4098-a2ae-651e7cd00567-serviceca\") pod \"node-ca-plvzg\" (UID: \"26f57ed5-a830-4098-a2ae-651e7cd00567\") " pod="openshift-image-registry/node-ca-plvzg"
Sep 10 17:13:18.840947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840641    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-cnibin\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.840947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840667    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbg2m\" (UniqueName: \"kubernetes.io/projected/7a5b4316-2391-4e12-9a80-12ace78ee49d-kube-api-access-bbg2m\") pod \"network-metrics-daemon-rzqqc\" (UID: \"7a5b4316-2391-4e12-9a80-12ace78ee49d\") " pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:18.840947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840699    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-sysctl-conf\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.840947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840723    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-run-netns\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.840947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840749    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-host-run-k8s-cni-cncf-io\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.840947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840776    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-host-run-multus-certs\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.840947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840799    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c56k\" (UniqueName: \"kubernetes.io/projected/2fa83261-0f33-4437-b34b-ccb301ff0ac9-kube-api-access-4c56k\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.840947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840822    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-kubernetes\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.840947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840843    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-run\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.840947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840869    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9efe55c6-4eef-4472-846d-4c3861daefa1-cni-binary-copy\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.840947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840894    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-run-openvswitch\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.840947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840920    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-run-ovn-kubernetes\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.840947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840943    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-modprobe-d\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.841749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840947    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7023d897-e2aa-40ea-be34-840f36db1f8d-ovnkube-script-lib\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.841749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840967    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret\") pod \"global-pull-secret-syncer-qqw5f\" (UID: \"5d278a9b-4d67-4eaf-a7a4-d2e980c2a775\") " pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:18.841749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.840995    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26f57ed5-a830-4098-a2ae-651e7cd00567-host\") pod \"node-ca-plvzg\" (UID: \"26f57ed5-a830-4098-a2ae-651e7cd00567\") " pod="openshift-image-registry/node-ca-plvzg"
Sep 10 17:13:18.841749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841019    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9efe55c6-4eef-4472-846d-4c3861daefa1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.841749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841045    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7023d897-e2aa-40ea-be34-840f36db1f8d-env-overrides\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.841749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841070    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbsnv\" (UniqueName: \"kubernetes.io/projected/7023d897-e2aa-40ea-be34-840f36db1f8d-kube-api-access-pbsnv\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.841749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841076    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-host-var-lib-cni-bin\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.841749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841105    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-system-cni-dir\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.841749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841156    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-multus-conf-dir\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.841749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841181    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69cd67eb-da77-4826-9bc2-a5202871c135-tmp-dir\") pod \"node-resolver-fl4lm\" (UID: \"69cd67eb-da77-4826-9bc2-a5202871c135\") " pod="openshift-dns/node-resolver-fl4lm"
Sep 10 17:13:18.841749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841192    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9efe55c6-4eef-4472-846d-4c3861daefa1-cnibin\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.841749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841232    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.841749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841283    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-cnibin\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.841749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841292    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-host-run-netns\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.841749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841320    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-host-var-lib-cni-multus\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.841749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841347    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs\") pod \"network-metrics-daemon-rzqqc\" (UID: \"7a5b4316-2391-4e12-9a80-12ace78ee49d\") " pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:18.841749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841377    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/47655ffd-db4a-4e92-a4f8-d170c8681d8c-agent-certs\") pod \"konnectivity-agent-mn5ts\" (UID: \"47655ffd-db4a-4e92-a4f8-d170c8681d8c\") " pod="kube-system/konnectivity-agent-mn5ts"
Sep 10 17:13:18.842538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841407    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98bnf\" (UniqueName: \"kubernetes.io/projected/85265614-4740-49c7-aa44-934ee4ea6fde-kube-api-access-98bnf\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.842538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841436    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-run-systemd\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.842538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841464    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-var-lib-openvswitch\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.842538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841489    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7023d897-e2aa-40ea-be34-840f36db1f8d-ovn-node-metrics-cert\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.842538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841514    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-multus-cni-dir\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.842538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841540    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-sysctl-d\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.842538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841566    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-sys\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.842538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841589    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-tmp\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.842538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841599    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-run-netns\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.842538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841615    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-kubelet-config\") pod \"global-pull-secret-syncer-qqw5f\" (UID: \"5d278a9b-4d67-4eaf-a7a4-d2e980c2a775\") " pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:18.842538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841643    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-cni-netd\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.842538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841649    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-host-run-k8s-cni-cncf-io\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.842538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841669    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfmn6\" (UniqueName: \"kubernetes.io/projected/26f57ed5-a830-4098-a2ae-651e7cd00567-kube-api-access-sfmn6\") pod \"node-ca-plvzg\" (UID: \"26f57ed5-a830-4098-a2ae-651e7cd00567\") " pod="openshift-image-registry/node-ca-plvzg"
Sep 10 17:13:18.842538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841699    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9efe55c6-4eef-4472-846d-4c3861daefa1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.842538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841710    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-host-run-multus-certs\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.842538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841726    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-slash\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.842538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841752    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-host-var-lib-kubelet\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.843360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841777    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-systemd\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.843360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841802    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-host\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.843360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841827    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf595\" (UniqueName: \"kubernetes.io/projected/0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb-kube-api-access-qf595\") pod \"iptables-alerter-qxx2h\" (UID: \"0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb\") " pod="openshift-network-operator/iptables-alerter-qxx2h"
Sep 10 17:13:18.843360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841854    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-systemd-units\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.843360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841878    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-node-log\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.843360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841907    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plszb\" (UniqueName: \"kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb\") pod \"network-check-target-l4rtn\" (UID: \"d993b686-b64a-4643-b53e-2021d0e476ee\") " pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:18.843360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841934    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f8vf\" (UniqueName: \"kubernetes.io/projected/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-kube-api-access-7f8vf\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.843360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841959    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/85265614-4740-49c7-aa44-934ee4ea6fde-etc-selinux\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.843360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.841988    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9efe55c6-4eef-4472-846d-4c3861daefa1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.843360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.842015    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-cni-bin\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.843360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.842042    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/85265614-4740-49c7-aa44-934ee4ea6fde-socket-dir\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.843360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.842067    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/85265614-4740-49c7-aa44-934ee4ea6fde-registration-dir\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.843360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.842095    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-run-ovn\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.843360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.842140    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-log-socket\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.843360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.842168    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-multus-socket-dir-parent\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.843360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.842197    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2fa83261-0f33-4437-b34b-ccb301ff0ac9-multus-daemon-config\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.843360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.842223    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-etc-kubernetes\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.844200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.842248    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/47655ffd-db4a-4e92-a4f8-d170c8681d8c-konnectivity-ca\") pod \"konnectivity-agent-mn5ts\" (UID: \"47655ffd-db4a-4e92-a4f8-d170c8681d8c\") " pod="kube-system/konnectivity-agent-mn5ts"
Sep 10 17:13:18.844200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.842276    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb-iptables-alerter-script\") pod \"iptables-alerter-qxx2h\" (UID: \"0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb\") " pod="openshift-network-operator/iptables-alerter-qxx2h"
Sep 10 17:13:18.844200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.842305    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9efe55c6-4eef-4472-846d-4c3861daefa1-system-cni-dir\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.844200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.842333    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-etc-openvswitch\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.844200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.842360    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-lib-modules\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.844200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.842386    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-var-lib-kubelet\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.844200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.842456    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9efe55c6-4eef-4472-846d-4c3861daefa1-cni-binary-copy\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.844200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.842520    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-run-openvswitch\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.844200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.842564    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-run-ovn-kubernetes\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.844200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.842627    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26f57ed5-a830-4098-a2ae-651e7cd00567-host\") pod \"node-ca-plvzg\" (UID: \"26f57ed5-a830-4098-a2ae-651e7cd00567\") " pod="openshift-image-registry/node-ca-plvzg"
Sep 10 17:13:18.844200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.843072    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9efe55c6-4eef-4472-846d-4c3861daefa1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.844200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.843158    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7023d897-e2aa-40ea-be34-840f36db1f8d-ovnkube-config\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.844200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.843457    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/26f57ed5-a830-4098-a2ae-651e7cd00567-serviceca\") pod \"node-ca-plvzg\" (UID: \"26f57ed5-a830-4098-a2ae-651e7cd00567\") " pod="openshift-image-registry/node-ca-plvzg"
Sep 10 17:13:18.844200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.843692    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2fa83261-0f33-4437-b34b-ccb301ff0ac9-cni-binary-copy\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.844200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.843753    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-hostroot\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.844200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.843788    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7023d897-e2aa-40ea-be34-840f36db1f8d-env-overrides\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.844200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.843807    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-kubelet\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.845805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.843862    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.845805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.843925    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-system-cni-dir\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.845805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.843961    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-multus-conf-dir\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.845805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.843993    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-host-run-netns\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.845805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.844015    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-host-var-lib-cni-multus\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.845805 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:18.844070    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:13:18.845805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.844116    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-systemd-units\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.845805 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:18.844154    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs podName:7a5b4316-2391-4e12-9a80-12ace78ee49d nodeName:}" failed. No retries permitted until 2026-09-10 17:13:19.344107575 +0000 UTC m=+3.025280516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs") pod "network-metrics-daemon-rzqqc" (UID: "7a5b4316-2391-4e12-9a80-12ace78ee49d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:13:18.845805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.844185    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-node-log\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.845805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.844219    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-run-systemd\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.845805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.844251    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-var-lib-openvswitch\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.845805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.844717    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-cni-bin\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.845805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.844777    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-log-socket\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.845805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.844820    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-run-ovn\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.845805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.844837    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-multus-cni-dir\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.845805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.844861    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-host-var-lib-kubelet\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.845805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.844934    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-cni-netd\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.845805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.844995    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-multus-socket-dir-parent\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.846531 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.845023    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2fa83261-0f33-4437-b34b-ccb301ff0ac9-multus-daemon-config\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.846531 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.845040    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-host-slash\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.846531 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.845043    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9efe55c6-4eef-4472-846d-4c3861daefa1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.846531 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.845089    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7023d897-e2aa-40ea-be34-840f36db1f8d-etc-openvswitch\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.846531 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.845155    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9efe55c6-4eef-4472-846d-4c3861daefa1-system-cni-dir\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.846531 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.845190    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fa83261-0f33-4437-b34b-ccb301ff0ac9-etc-kubernetes\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.846531 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.845274    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 17:13:18.846531 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.845626    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9efe55c6-4eef-4472-846d-4c3861daefa1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.850414 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.850390    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7023d897-e2aa-40ea-be34-840f36db1f8d-ovn-node-metrics-cert\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.850734 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.850717    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c56k\" (UniqueName: \"kubernetes.io/projected/2fa83261-0f33-4437-b34b-ccb301ff0ac9-kube-api-access-4c56k\") pod \"multus-gmg9v\" (UID: \"2fa83261-0f33-4437-b34b-ccb301ff0ac9\") " pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:18.850845 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.850817    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbg2m\" (UniqueName: \"kubernetes.io/projected/7a5b4316-2391-4e12-9a80-12ace78ee49d-kube-api-access-bbg2m\") pod \"network-metrics-daemon-rzqqc\" (UID: \"7a5b4316-2391-4e12-9a80-12ace78ee49d\") " pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:18.851721 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.851699    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhtxp\" (UniqueName: \"kubernetes.io/projected/9efe55c6-4eef-4472-846d-4c3861daefa1-kube-api-access-nhtxp\") pod \"multus-additional-cni-plugins-tp7gl\" (UID: \"9efe55c6-4eef-4472-846d-4c3861daefa1\") " pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:18.852030 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.852009    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbsnv\" (UniqueName: \"kubernetes.io/projected/7023d897-e2aa-40ea-be34-840f36db1f8d-kube-api-access-pbsnv\") pod \"ovnkube-node-ssbcl\" (UID: \"7023d897-e2aa-40ea-be34-840f36db1f8d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:18.859554 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:18.859507    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:13:18.859692 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:18.859562    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:13:18.859692 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:18.859582    2566 projected.go:196] Error preparing data for projected volume kube-api-access-plszb for pod openshift-network-diagnostics/network-check-target-l4rtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:13:18.859692 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:18.859675    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb podName:d993b686-b64a-4643-b53e-2021d0e476ee nodeName:}" failed. No retries permitted until 2026-09-10 17:13:19.359646897 +0000 UTC m=+3.040819836 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-plszb" (UniqueName: "kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb") pod "network-check-target-l4rtn" (UID: "d993b686-b64a-4643-b53e-2021d0e476ee") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:13:18.863892 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.863861    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfmn6\" (UniqueName: \"kubernetes.io/projected/26f57ed5-a830-4098-a2ae-651e7cd00567-kube-api-access-sfmn6\") pod \"node-ca-plvzg\" (UID: \"26f57ed5-a830-4098-a2ae-651e7cd00567\") " pod="openshift-image-registry/node-ca-plvzg"
Sep 10 17:13:18.942866 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.942818    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-systemd\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.943079 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.942877    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-host\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.943079 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.942901    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf595\" (UniqueName: \"kubernetes.io/projected/0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb-kube-api-access-qf595\") pod \"iptables-alerter-qxx2h\" (UID: \"0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb\") " pod="openshift-network-operator/iptables-alerter-qxx2h"
Sep 10 17:13:18.943079 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.942941    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f8vf\" (UniqueName: \"kubernetes.io/projected/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-kube-api-access-7f8vf\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.943079 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.942944    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-systemd\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.943079 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.942964    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/85265614-4740-49c7-aa44-934ee4ea6fde-etc-selinux\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.943079 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.942988    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/85265614-4740-49c7-aa44-934ee4ea6fde-socket-dir\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.943079 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943010    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/85265614-4740-49c7-aa44-934ee4ea6fde-registration-dir\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.943079 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943038    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/47655ffd-db4a-4e92-a4f8-d170c8681d8c-konnectivity-ca\") pod \"konnectivity-agent-mn5ts\" (UID: \"47655ffd-db4a-4e92-a4f8-d170c8681d8c\") " pod="kube-system/konnectivity-agent-mn5ts"
Sep 10 17:13:18.943079 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943061    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb-iptables-alerter-script\") pod \"iptables-alerter-qxx2h\" (UID: \"0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb\") " pod="openshift-network-operator/iptables-alerter-qxx2h"
Sep 10 17:13:18.943079 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943083    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-lib-modules\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.943638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943105    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-var-lib-kubelet\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.943638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943142    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-tuned\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.943638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943170    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/85265614-4740-49c7-aa44-934ee4ea6fde-sys-fs\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.943638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943197    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzfk7\" (UniqueName: \"kubernetes.io/projected/69cd67eb-da77-4826-9bc2-a5202871c135-kube-api-access-nzfk7\") pod \"node-resolver-fl4lm\" (UID: \"69cd67eb-da77-4826-9bc2-a5202871c135\") " pod="openshift-dns/node-resolver-fl4lm"
Sep 10 17:13:18.943638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943232    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb-host-slash\") pod \"iptables-alerter-qxx2h\" (UID: \"0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb\") " pod="openshift-network-operator/iptables-alerter-qxx2h"
Sep 10 17:13:18.943638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943245    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-host\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.943638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943254    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-dbus\") pod \"global-pull-secret-syncer-qqw5f\" (UID: \"5d278a9b-4d67-4eaf-a7a4-d2e980c2a775\") " pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:18.943638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943281    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/85265614-4740-49c7-aa44-934ee4ea6fde-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.943638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943300    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/85265614-4740-49c7-aa44-934ee4ea6fde-device-dir\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.943638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943322    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/69cd67eb-da77-4826-9bc2-a5202871c135-hosts-file\") pod \"node-resolver-fl4lm\" (UID: \"69cd67eb-da77-4826-9bc2-a5202871c135\") " pod="openshift-dns/node-resolver-fl4lm"
Sep 10 17:13:18.943638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943342    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-sysconfig\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.943638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943363    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-lib-modules\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.943638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943372    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-sysctl-conf\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.943638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943401    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-kubernetes\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.943638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943423    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-run\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.943638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943448    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-modprobe-d\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.943638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943474    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret\") pod \"global-pull-secret-syncer-qqw5f\" (UID: \"5d278a9b-4d67-4eaf-a7a4-d2e980c2a775\") " pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:18.944477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943507    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69cd67eb-da77-4826-9bc2-a5202871c135-tmp-dir\") pod \"node-resolver-fl4lm\" (UID: \"69cd67eb-da77-4826-9bc2-a5202871c135\") " pod="openshift-dns/node-resolver-fl4lm"
Sep 10 17:13:18.944477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943582    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-var-lib-kubelet\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.944477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943619    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/85265614-4740-49c7-aa44-934ee4ea6fde-registration-dir\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.944477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943672    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/47655ffd-db4a-4e92-a4f8-d170c8681d8c-agent-certs\") pod \"konnectivity-agent-mn5ts\" (UID: \"47655ffd-db4a-4e92-a4f8-d170c8681d8c\") " pod="kube-system/konnectivity-agent-mn5ts"
Sep 10 17:13:18.944477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943701    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98bnf\" (UniqueName: \"kubernetes.io/projected/85265614-4740-49c7-aa44-934ee4ea6fde-kube-api-access-98bnf\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.944477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943734    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-sysctl-d\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.944477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943761    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-sys\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.944477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943783    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-tmp\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.944477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943809    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-kubelet-config\") pod \"global-pull-secret-syncer-qqw5f\" (UID: \"5d278a9b-4d67-4eaf-a7a4-d2e980c2a775\") " pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:18.944477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943897    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-kubelet-config\") pod \"global-pull-secret-syncer-qqw5f\" (UID: \"5d278a9b-4d67-4eaf-a7a4-d2e980c2a775\") " pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:18.944477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.943948    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/85265614-4740-49c7-aa44-934ee4ea6fde-etc-selinux\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.944477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.944117    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/47655ffd-db4a-4e92-a4f8-d170c8681d8c-konnectivity-ca\") pod \"konnectivity-agent-mn5ts\" (UID: \"47655ffd-db4a-4e92-a4f8-d170c8681d8c\") " pod="kube-system/konnectivity-agent-mn5ts"
Sep 10 17:13:18.944477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.944139    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/85265614-4740-49c7-aa44-934ee4ea6fde-socket-dir\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.944477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.944119    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb-iptables-alerter-script\") pod \"iptables-alerter-qxx2h\" (UID: \"0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb\") " pod="openshift-network-operator/iptables-alerter-qxx2h"
Sep 10 17:13:18.944477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.944259    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/69cd67eb-da77-4826-9bc2-a5202871c135-hosts-file\") pod \"node-resolver-fl4lm\" (UID: \"69cd67eb-da77-4826-9bc2-a5202871c135\") " pod="openshift-dns/node-resolver-fl4lm"
Sep 10 17:13:18.944477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.944388    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69cd67eb-da77-4826-9bc2-a5202871c135-tmp-dir\") pod \"node-resolver-fl4lm\" (UID: \"69cd67eb-da77-4826-9bc2-a5202871c135\") " pod="openshift-dns/node-resolver-fl4lm"
Sep 10 17:13:18.945236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.944488    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-sysctl-d\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.945236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.944490    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb-host-slash\") pod \"iptables-alerter-qxx2h\" (UID: \"0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb\") " pod="openshift-network-operator/iptables-alerter-qxx2h"
Sep 10 17:13:18.945236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.944522    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/85265614-4740-49c7-aa44-934ee4ea6fde-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.945236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.944543    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/85265614-4740-49c7-aa44-934ee4ea6fde-sys-fs\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.945236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.944555    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-sys\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.945236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.944584    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-dbus\") pod \"global-pull-secret-syncer-qqw5f\" (UID: \"5d278a9b-4d67-4eaf-a7a4-d2e980c2a775\") " pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:18.945236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.944633    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/85265614-4740-49c7-aa44-934ee4ea6fde-device-dir\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.945236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.944679    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-kubernetes\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.945236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.944721    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-sysconfig\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.945236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.944814    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-modprobe-d\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.945236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.944850    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-sysctl-conf\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.945236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.944858    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-run\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.945236 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:18.944936    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:13:18.945236 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:18.944987    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret podName:5d278a9b-4d67-4eaf-a7a4-d2e980c2a775 nodeName:}" failed. No retries permitted until 2026-09-10 17:13:19.444970549 +0000 UTC m=+3.126143479 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret") pod "global-pull-secret-syncer-qqw5f" (UID: "5d278a9b-4d67-4eaf-a7a4-d2e980c2a775") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:13:18.947452 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.947421    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-etc-tuned\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.947678 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.947655    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/47655ffd-db4a-4e92-a4f8-d170c8681d8c-agent-certs\") pod \"konnectivity-agent-mn5ts\" (UID: \"47655ffd-db4a-4e92-a4f8-d170c8681d8c\") " pod="kube-system/konnectivity-agent-mn5ts"
Sep 10 17:13:18.947756 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.947665    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-tmp\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.951927 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.951890    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf595\" (UniqueName: \"kubernetes.io/projected/0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb-kube-api-access-qf595\") pod \"iptables-alerter-qxx2h\" (UID: \"0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb\") " pod="openshift-network-operator/iptables-alerter-qxx2h"
Sep 10 17:13:18.952682 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.952654    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98bnf\" (UniqueName: \"kubernetes.io/projected/85265614-4740-49c7-aa44-934ee4ea6fde-kube-api-access-98bnf\") pod \"aws-ebs-csi-driver-node-c26xn\" (UID: \"85265614-4740-49c7-aa44-934ee4ea6fde\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:18.953099 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.953025    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f8vf\" (UniqueName: \"kubernetes.io/projected/d29a5496-59b7-42ab-8f4c-b3dbbf771a27-kube-api-access-7f8vf\") pod \"tuned-nsv5g\" (UID: \"d29a5496-59b7-42ab-8f4c-b3dbbf771a27\") " pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:18.954415 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:18.954388    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzfk7\" (UniqueName: \"kubernetes.io/projected/69cd67eb-da77-4826-9bc2-a5202871c135-kube-api-access-nzfk7\") pod \"node-resolver-fl4lm\" (UID: \"69cd67eb-da77-4826-9bc2-a5202871c135\") " pod="openshift-dns/node-resolver-fl4lm"
Sep 10 17:13:19.029644 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:19.029603    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gmg9v"
Sep 10 17:13:19.041662 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:19.041627    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-plvzg"
Sep 10 17:13:19.052592 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:19.052558    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tp7gl"
Sep 10 17:13:19.058555 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:19.058531    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:19.068225 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:19.068200    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qxx2h"
Sep 10 17:13:19.076992 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:19.076967    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mn5ts"
Sep 10 17:13:19.085791 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:19.085766    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nsv5g"
Sep 10 17:13:19.094592 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:19.094567    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fl4lm"
Sep 10 17:13:19.101575 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:19.101546    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn"
Sep 10 17:13:19.346335 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:19.346295    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs\") pod \"network-metrics-daemon-rzqqc\" (UID: \"7a5b4316-2391-4e12-9a80-12ace78ee49d\") " pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:19.346516 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:19.346489    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:13:19.346592 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:19.346577    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs podName:7a5b4316-2391-4e12-9a80-12ace78ee49d nodeName:}" failed. No retries permitted until 2026-09-10 17:13:20.346553856 +0000 UTC m=+4.027726781 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs") pod "network-metrics-daemon-rzqqc" (UID: "7a5b4316-2391-4e12-9a80-12ace78ee49d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:13:19.447356 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:19.447317    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret\") pod \"global-pull-secret-syncer-qqw5f\" (UID: \"5d278a9b-4d67-4eaf-a7a4-d2e980c2a775\") " pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:19.447501 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:19.447373    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plszb\" (UniqueName: \"kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb\") pod \"network-check-target-l4rtn\" (UID: \"d993b686-b64a-4643-b53e-2021d0e476ee\") " pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:19.447501 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:19.447486    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:13:19.447579 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:19.447503    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:13:19.447579 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:19.447513    2566 projected.go:196] Error preparing data for projected volume kube-api-access-plszb for pod openshift-network-diagnostics/network-check-target-l4rtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:13:19.447579 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:19.447487    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:13:19.447659 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:19.447563    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb podName:d993b686-b64a-4643-b53e-2021d0e476ee nodeName:}" failed. No retries permitted until 2026-09-10 17:13:20.447548354 +0000 UTC m=+4.128721274 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-plszb" (UniqueName: "kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb") pod "network-check-target-l4rtn" (UID: "d993b686-b64a-4643-b53e-2021d0e476ee") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:13:19.447659 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:19.447622    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret podName:5d278a9b-4d67-4eaf-a7a4-d2e980c2a775 nodeName:}" failed. No retries permitted until 2026-09-10 17:13:20.447608301 +0000 UTC m=+4.128781222 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret") pod "global-pull-secret-syncer-qqw5f" (UID: "5d278a9b-4d67-4eaf-a7a4-d2e980c2a775") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:13:19.775922 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:19.775880    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 17:08:18 +0000 UTC" deadline="2028-02-18 21:29:31.765547224 +0000 UTC"
Sep 10 17:13:19.775922 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:19.775918    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12628h16m11.989634172s"
Sep 10 17:13:19.820895 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:19.820864    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9efe55c6_4eef_4472_846d_4c3861daefa1.slice/crio-953fae00fd265beea60599847dcd05d64eeba9e31d653aed6e36c191d0da4fb1 WatchSource:0}: Error finding container 953fae00fd265beea60599847dcd05d64eeba9e31d653aed6e36c191d0da4fb1: Status 404 returned error can't find the container with id 953fae00fd265beea60599847dcd05d64eeba9e31d653aed6e36c191d0da4fb1
Sep 10 17:13:19.838092 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:19.838057    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tp7gl" event={"ID":"9efe55c6-4eef-4472-846d-4c3861daefa1","Type":"ContainerStarted","Data":"953fae00fd265beea60599847dcd05d64eeba9e31d653aed6e36c191d0da4fb1"}
Sep 10 17:13:19.843993 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:19.843808    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85265614_4740_49c7_aa44_934ee4ea6fde.slice/crio-4e2effbb078d502fe61f04fee27f35668a979548e152672db86c40f053f70933 WatchSource:0}: Error finding container 4e2effbb078d502fe61f04fee27f35668a979548e152672db86c40f053f70933: Status 404 returned error can't find the container with id 4e2effbb078d502fe61f04fee27f35668a979548e152672db86c40f053f70933
Sep 10 17:13:19.844497 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:19.844462    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd29a5496_59b7_42ab_8f4c_b3dbbf771a27.slice/crio-13d1fec2a5befb5d8bba0b81c33e74e77592b999b62780eb9e34abc60b96e001 WatchSource:0}: Error finding container 13d1fec2a5befb5d8bba0b81c33e74e77592b999b62780eb9e34abc60b96e001: Status 404 returned error can't find the container with id 13d1fec2a5befb5d8bba0b81c33e74e77592b999b62780eb9e34abc60b96e001
Sep 10 17:13:19.846309 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:19.846281    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47655ffd_db4a_4e92_a4f8_d170c8681d8c.slice/crio-16b6eb66e3058e80dea1fff4574afeb60cd12d59ac35f64e9668c235877703f6 WatchSource:0}: Error finding container 16b6eb66e3058e80dea1fff4574afeb60cd12d59ac35f64e9668c235877703f6: Status 404 returned error can't find the container with id 16b6eb66e3058e80dea1fff4574afeb60cd12d59ac35f64e9668c235877703f6
Sep 10 17:13:19.851840 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:19.851816    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69cd67eb_da77_4826_9bc2_a5202871c135.slice/crio-aac6122f6bdeaf7cf9abd39b9c6e59ced4046026737cff3d537722fe5059adbd WatchSource:0}: Error finding container aac6122f6bdeaf7cf9abd39b9c6e59ced4046026737cff3d537722fe5059adbd: Status 404 returned error can't find the container with id aac6122f6bdeaf7cf9abd39b9c6e59ced4046026737cff3d537722fe5059adbd
Sep 10 17:13:19.852969 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:19.852943    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fa83261_0f33_4437_b34b_ccb301ff0ac9.slice/crio-9cf2fb6f46b6d98d2493ae396efea22acfe83950a3c1ae330f2733bcb61f988d WatchSource:0}: Error finding container 9cf2fb6f46b6d98d2493ae396efea22acfe83950a3c1ae330f2733bcb61f988d: Status 404 returned error can't find the container with id 9cf2fb6f46b6d98d2493ae396efea22acfe83950a3c1ae330f2733bcb61f988d
Sep 10 17:13:19.853962 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:19.853923    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26f57ed5_a830_4098_a2ae_651e7cd00567.slice/crio-46566dab3254aa1bc0b74817e7db7f8077eabc2073d77dfbd5a2632f7a77a611 WatchSource:0}: Error finding container 46566dab3254aa1bc0b74817e7db7f8077eabc2073d77dfbd5a2632f7a77a611: Status 404 returned error can't find the container with id 46566dab3254aa1bc0b74817e7db7f8077eabc2073d77dfbd5a2632f7a77a611
Sep 10 17:13:19.855322 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:19.855299    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c9c654e_8e9b_43a5_b3fc_5afc8a0e81eb.slice/crio-72b55441a3662151e3eda772f5eaa59566fb55f28d0e3a4cf9aa843a663e1215 WatchSource:0}: Error finding container 72b55441a3662151e3eda772f5eaa59566fb55f28d0e3a4cf9aa843a663e1215: Status 404 returned error can't find the container with id 72b55441a3662151e3eda772f5eaa59566fb55f28d0e3a4cf9aa843a663e1215
Sep 10 17:13:19.855764 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:19.855740    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7023d897_e2aa_40ea_be34_840f36db1f8d.slice/crio-8d83ed8b7299007ae3dc13de6a5e22bbcee7b13b2bb10d364bdf015260bd08b8 WatchSource:0}: Error finding container 8d83ed8b7299007ae3dc13de6a5e22bbcee7b13b2bb10d364bdf015260bd08b8: Status 404 returned error can't find the container with id 8d83ed8b7299007ae3dc13de6a5e22bbcee7b13b2bb10d364bdf015260bd08b8
Sep 10 17:13:20.355146 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.355034    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs\") pod \"network-metrics-daemon-rzqqc\" (UID: \"7a5b4316-2391-4e12-9a80-12ace78ee49d\") " pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:20.355333 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:20.355205    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:13:20.355333 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:20.355281    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs podName:7a5b4316-2391-4e12-9a80-12ace78ee49d nodeName:}" failed. No retries permitted until 2026-09-10 17:13:22.355260197 +0000 UTC m=+6.036433118 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs") pod "network-metrics-daemon-rzqqc" (UID: "7a5b4316-2391-4e12-9a80-12ace78ee49d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:13:20.456655 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.455810    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret\") pod \"global-pull-secret-syncer-qqw5f\" (UID: \"5d278a9b-4d67-4eaf-a7a4-d2e980c2a775\") " pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:20.456655 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.455890    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plszb\" (UniqueName: \"kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb\") pod \"network-check-target-l4rtn\" (UID: \"d993b686-b64a-4643-b53e-2021d0e476ee\") " pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:20.456655 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:20.456036    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:13:20.456655 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:20.456054    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:13:20.456655 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:20.456067    2566 projected.go:196] Error preparing data for projected volume kube-api-access-plszb for pod openshift-network-diagnostics/network-check-target-l4rtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:13:20.456655 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:20.456141    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb podName:d993b686-b64a-4643-b53e-2021d0e476ee nodeName:}" failed. No retries permitted until 2026-09-10 17:13:22.456107842 +0000 UTC m=+6.137280767 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-plszb" (UniqueName: "kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb") pod "network-check-target-l4rtn" (UID: "d993b686-b64a-4643-b53e-2021d0e476ee") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:13:20.456655 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:20.456560    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:13:20.456655 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:20.456614    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret podName:5d278a9b-4d67-4eaf-a7a4-d2e980c2a775 nodeName:}" failed. No retries permitted until 2026-09-10 17:13:22.456598125 +0000 UTC m=+6.137771051 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret") pod "global-pull-secret-syncer-qqw5f" (UID: "5d278a9b-4d67-4eaf-a7a4-d2e980c2a775") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:13:20.777001 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.776932    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 17:08:18 +0000 UTC" deadline="2028-03-06 03:37:48.944740048 +0000 UTC"
Sep 10 17:13:20.777001 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.776995    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13018h24m28.167748749s"
Sep 10 17:13:20.837016 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.829118    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:20.837016 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:20.829294    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4rtn" podUID="d993b686-b64a-4643-b53e-2021d0e476ee"
Sep 10 17:13:20.837016 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.829771    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:20.837016 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:20.829840    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqw5f" podUID="5d278a9b-4d67-4eaf-a7a4-d2e980c2a775"
Sep 10 17:13:20.837016 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.829903    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:20.837016 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:20.829950    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzqqc" podUID="7a5b4316-2391-4e12-9a80-12ace78ee49d"
Sep 10 17:13:20.846454 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.846388    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gmg9v" event={"ID":"2fa83261-0f33-4437-b34b-ccb301ff0ac9","Type":"ContainerStarted","Data":"9cf2fb6f46b6d98d2493ae396efea22acfe83950a3c1ae330f2733bcb61f988d"}
Sep 10 17:13:20.851706 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.851498    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-plvzg" event={"ID":"26f57ed5-a830-4098-a2ae-651e7cd00567","Type":"ContainerStarted","Data":"46566dab3254aa1bc0b74817e7db7f8077eabc2073d77dfbd5a2632f7a77a611"}
Sep 10 17:13:20.859418 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.859378    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nsv5g" event={"ID":"d29a5496-59b7-42ab-8f4c-b3dbbf771a27","Type":"ContainerStarted","Data":"13d1fec2a5befb5d8bba0b81c33e74e77592b999b62780eb9e34abc60b96e001"}
Sep 10 17:13:20.865467 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.865426    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn" event={"ID":"85265614-4740-49c7-aa44-934ee4ea6fde","Type":"ContainerStarted","Data":"4e2effbb078d502fe61f04fee27f35668a979548e152672db86c40f053f70933"}
Sep 10 17:13:20.884186 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.883784    2566 generic.go:356] "Generic (PLEG): container finished" podID="cd25bbcec769cf1adcfbe899ed035566" containerID="612eec97a284d89182ad3cb9806dba71f44ff946e15b1ee1ddcce707f0f2110d" exitCode=0
Sep 10 17:13:20.884378 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.884314    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal" event={"ID":"cd25bbcec769cf1adcfbe899ed035566","Type":"ContainerDied","Data":"612eec97a284d89182ad3cb9806dba71f44ff946e15b1ee1ddcce707f0f2110d"}
Sep 10 17:13:20.892323 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.892201    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fl4lm" event={"ID":"69cd67eb-da77-4826-9bc2-a5202871c135","Type":"ContainerStarted","Data":"aac6122f6bdeaf7cf9abd39b9c6e59ced4046026737cff3d537722fe5059adbd"}
Sep 10 17:13:20.910139 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.910089    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mn5ts" event={"ID":"47655ffd-db4a-4e92-a4f8-d170c8681d8c","Type":"ContainerStarted","Data":"16b6eb66e3058e80dea1fff4574afeb60cd12d59ac35f64e9668c235877703f6"}
Sep 10 17:13:20.940748 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.940702    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-123.ec2.internal" event={"ID":"b7691c42ef3bfce427afbddd33d4f4b3","Type":"ContainerStarted","Data":"e4fcd34f1c9300a3b0a1f486e1551f1200609b1d08f2c1acfc9b45574c465337"}
Sep 10 17:13:20.956630 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.956557    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-123.ec2.internal" podStartSLOduration=3.956533282 podStartE2EDuration="3.956533282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:13:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:13:20.956352649 +0000 UTC m=+4.637525608" watchObservedRunningTime="2026-09-10 17:13:20.956533282 +0000 UTC m=+4.637706226"
Sep 10 17:13:20.958542 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.958423    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl" event={"ID":"7023d897-e2aa-40ea-be34-840f36db1f8d","Type":"ContainerStarted","Data":"8d83ed8b7299007ae3dc13de6a5e22bbcee7b13b2bb10d364bdf015260bd08b8"}
Sep 10 17:13:20.966329 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:20.966202    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qxx2h" event={"ID":"0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb","Type":"ContainerStarted","Data":"72b55441a3662151e3eda772f5eaa59566fb55f28d0e3a4cf9aa843a663e1215"}
Sep 10 17:13:21.993227 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:21.993188    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal" event={"ID":"cd25bbcec769cf1adcfbe899ed035566","Type":"ContainerStarted","Data":"691205b86c7b0b6236c7e4c2dd1cfaba0c4ce3df4f67077cf5c597c0284e9c9c"}
Sep 10 17:13:22.371158 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:22.370478    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs\") pod \"network-metrics-daemon-rzqqc\" (UID: \"7a5b4316-2391-4e12-9a80-12ace78ee49d\") " pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:22.371158 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:22.370675    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:13:22.371158 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:22.370742    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs podName:7a5b4316-2391-4e12-9a80-12ace78ee49d nodeName:}" failed. No retries permitted until 2026-09-10 17:13:26.370722881 +0000 UTC m=+10.051895806 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs") pod "network-metrics-daemon-rzqqc" (UID: "7a5b4316-2391-4e12-9a80-12ace78ee49d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:13:22.472380 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:22.471518    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret\") pod \"global-pull-secret-syncer-qqw5f\" (UID: \"5d278a9b-4d67-4eaf-a7a4-d2e980c2a775\") " pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:22.472380 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:22.471604    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plszb\" (UniqueName: \"kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb\") pod \"network-check-target-l4rtn\" (UID: \"d993b686-b64a-4643-b53e-2021d0e476ee\") " pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:22.472380 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:22.471761    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:13:22.472380 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:22.471793    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:13:22.472380 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:22.471807    2566 projected.go:196] Error preparing data for projected volume kube-api-access-plszb for pod openshift-network-diagnostics/network-check-target-l4rtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:13:22.472380 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:22.471871    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb podName:d993b686-b64a-4643-b53e-2021d0e476ee nodeName:}" failed. No retries permitted until 2026-09-10 17:13:26.471852761 +0000 UTC m=+10.153025686 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-plszb" (UniqueName: "kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb") pod "network-check-target-l4rtn" (UID: "d993b686-b64a-4643-b53e-2021d0e476ee") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:13:22.472380 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:22.472291    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:13:22.479617 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:22.473501    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret podName:5d278a9b-4d67-4eaf-a7a4-d2e980c2a775 nodeName:}" failed. No retries permitted until 2026-09-10 17:13:26.472327179 +0000 UTC m=+10.153500105 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret") pod "global-pull-secret-syncer-qqw5f" (UID: "5d278a9b-4d67-4eaf-a7a4-d2e980c2a775") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:13:22.829313 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:22.828594    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:22.829313 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:22.828733    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqw5f" podUID="5d278a9b-4d67-4eaf-a7a4-d2e980c2a775"
Sep 10 17:13:22.829313 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:22.829144    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:22.829313 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:22.829256    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzqqc" podUID="7a5b4316-2391-4e12-9a80-12ace78ee49d"
Sep 10 17:13:22.829763 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:22.829733    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:22.829852 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:22.829832    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4rtn" podUID="d993b686-b64a-4643-b53e-2021d0e476ee"
Sep 10 17:13:24.827806 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:24.827763    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:24.828510 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:24.827996    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqw5f" podUID="5d278a9b-4d67-4eaf-a7a4-d2e980c2a775"
Sep 10 17:13:24.828510 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:24.827763    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:24.828510 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:24.827763    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:24.828510 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:24.828306    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzqqc" podUID="7a5b4316-2391-4e12-9a80-12ace78ee49d"
Sep 10 17:13:24.828510 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:24.828156    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4rtn" podUID="d993b686-b64a-4643-b53e-2021d0e476ee"
Sep 10 17:13:26.405297 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:26.405254    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs\") pod \"network-metrics-daemon-rzqqc\" (UID: \"7a5b4316-2391-4e12-9a80-12ace78ee49d\") " pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:26.405789 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:26.405466    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:13:26.405789 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:26.405534    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs podName:7a5b4316-2391-4e12-9a80-12ace78ee49d nodeName:}" failed. No retries permitted until 2026-09-10 17:13:34.405514713 +0000 UTC m=+18.086687638 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs") pod "network-metrics-daemon-rzqqc" (UID: "7a5b4316-2391-4e12-9a80-12ace78ee49d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:13:26.506330 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:26.506285    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret\") pod \"global-pull-secret-syncer-qqw5f\" (UID: \"5d278a9b-4d67-4eaf-a7a4-d2e980c2a775\") " pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:26.506528 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:26.506362    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plszb\" (UniqueName: \"kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb\") pod \"network-check-target-l4rtn\" (UID: \"d993b686-b64a-4643-b53e-2021d0e476ee\") " pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:26.506528 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:26.506503    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:13:26.506528 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:26.506520    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:13:26.506698 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:26.506533    2566 projected.go:196] Error preparing data for projected volume kube-api-access-plszb for pod openshift-network-diagnostics/network-check-target-l4rtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:13:26.506698 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:26.506597    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb podName:d993b686-b64a-4643-b53e-2021d0e476ee nodeName:}" failed. No retries permitted until 2026-09-10 17:13:34.506579288 +0000 UTC m=+18.187752208 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-plszb" (UniqueName: "kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb") pod "network-check-target-l4rtn" (UID: "d993b686-b64a-4643-b53e-2021d0e476ee") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:13:26.506698 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:26.506654    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:13:26.506698 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:26.506696    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret podName:5d278a9b-4d67-4eaf-a7a4-d2e980c2a775 nodeName:}" failed. No retries permitted until 2026-09-10 17:13:34.506683656 +0000 UTC m=+18.187856581 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret") pod "global-pull-secret-syncer-qqw5f" (UID: "5d278a9b-4d67-4eaf-a7a4-d2e980c2a775") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:13:26.831143 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:26.831090    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:26.831520 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:26.831501    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:26.831751 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:26.831722    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4rtn" podUID="d993b686-b64a-4643-b53e-2021d0e476ee"
Sep 10 17:13:26.832322 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:26.832285    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqw5f" podUID="5d278a9b-4d67-4eaf-a7a4-d2e980c2a775"
Sep 10 17:13:26.832435 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:26.832363    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:26.832513 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:26.832485    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzqqc" podUID="7a5b4316-2391-4e12-9a80-12ace78ee49d"
Sep 10 17:13:28.828468 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:28.828429    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:28.828468 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:28.828465    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:28.829056 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:28.828435    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:28.829056 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:28.828608    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqw5f" podUID="5d278a9b-4d67-4eaf-a7a4-d2e980c2a775"
Sep 10 17:13:28.829056 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:28.828680    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzqqc" podUID="7a5b4316-2391-4e12-9a80-12ace78ee49d"
Sep 10 17:13:28.829056 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:28.828774    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4rtn" podUID="d993b686-b64a-4643-b53e-2021d0e476ee"
Sep 10 17:13:30.828339 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:30.828299    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:30.828813 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:30.828348    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:30.828813 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:30.828310    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:30.828813 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:30.828446    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqw5f" podUID="5d278a9b-4d67-4eaf-a7a4-d2e980c2a775"
Sep 10 17:13:30.828813 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:30.828507    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4rtn" podUID="d993b686-b64a-4643-b53e-2021d0e476ee"
Sep 10 17:13:30.828813 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:30.828601    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzqqc" podUID="7a5b4316-2391-4e12-9a80-12ace78ee49d"
Sep 10 17:13:32.828229 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:32.828003    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:32.828712 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:32.828018    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:32.828712 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:32.828315    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqw5f" podUID="5d278a9b-4d67-4eaf-a7a4-d2e980c2a775"
Sep 10 17:13:32.828712 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:32.828424    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzqqc" podUID="7a5b4316-2391-4e12-9a80-12ace78ee49d"
Sep 10 17:13:32.828712 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:32.828073    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:32.828712 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:32.828538    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4rtn" podUID="d993b686-b64a-4643-b53e-2021d0e476ee"
Sep 10 17:13:34.466281 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:34.466154    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs\") pod \"network-metrics-daemon-rzqqc\" (UID: \"7a5b4316-2391-4e12-9a80-12ace78ee49d\") " pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:34.466731 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:34.466329    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:13:34.466731 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:34.466407    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs podName:7a5b4316-2391-4e12-9a80-12ace78ee49d nodeName:}" failed. No retries permitted until 2026-09-10 17:13:50.466384654 +0000 UTC m=+34.147557578 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs") pod "network-metrics-daemon-rzqqc" (UID: "7a5b4316-2391-4e12-9a80-12ace78ee49d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:13:34.567159 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:34.567101    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret\") pod \"global-pull-secret-syncer-qqw5f\" (UID: \"5d278a9b-4d67-4eaf-a7a4-d2e980c2a775\") " pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:34.567358 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:34.567213    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plszb\" (UniqueName: \"kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb\") pod \"network-check-target-l4rtn\" (UID: \"d993b686-b64a-4643-b53e-2021d0e476ee\") " pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:34.567358 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:34.567267    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:13:34.567358 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:34.567350    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret podName:5d278a9b-4d67-4eaf-a7a4-d2e980c2a775 nodeName:}" failed. No retries permitted until 2026-09-10 17:13:50.56733072 +0000 UTC m=+34.248503645 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret") pod "global-pull-secret-syncer-qqw5f" (UID: "5d278a9b-4d67-4eaf-a7a4-d2e980c2a775") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:13:34.567522 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:34.567365    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:13:34.567522 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:34.567382    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:13:34.567522 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:34.567392    2566 projected.go:196] Error preparing data for projected volume kube-api-access-plszb for pod openshift-network-diagnostics/network-check-target-l4rtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:13:34.567522 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:34.567447    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb podName:d993b686-b64a-4643-b53e-2021d0e476ee nodeName:}" failed. No retries permitted until 2026-09-10 17:13:50.567431073 +0000 UTC m=+34.248603997 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-plszb" (UniqueName: "kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb") pod "network-check-target-l4rtn" (UID: "d993b686-b64a-4643-b53e-2021d0e476ee") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:13:34.828365 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:34.828324    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:34.828576 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:34.828328    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:34.828576 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:34.828469    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqw5f" podUID="5d278a9b-4d67-4eaf-a7a4-d2e980c2a775"
Sep 10 17:13:34.828576 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:34.828335    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:34.828738 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:34.828589    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzqqc" podUID="7a5b4316-2391-4e12-9a80-12ace78ee49d"
Sep 10 17:13:34.828738 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:34.828676    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4rtn" podUID="d993b686-b64a-4643-b53e-2021d0e476ee"
Sep 10 17:13:36.828445 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:36.828403    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:36.828929 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:36.828532    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:36.828929 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:36.828544    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqw5f" podUID="5d278a9b-4d67-4eaf-a7a4-d2e980c2a775"
Sep 10 17:13:36.828929 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:36.828671    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:36.828929 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:36.828689    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzqqc" podUID="7a5b4316-2391-4e12-9a80-12ace78ee49d"
Sep 10 17:13:36.828929 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:36.828751    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4rtn" podUID="d993b686-b64a-4643-b53e-2021d0e476ee"
Sep 10 17:13:38.027208 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:38.027119    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gmg9v" event={"ID":"2fa83261-0f33-4437-b34b-ccb301ff0ac9","Type":"ContainerStarted","Data":"d79cba49dd524c4f6a8b7ab3107450ab04898431ff4ba3d859f3c43e2d6b98ea"}
Sep 10 17:13:38.029441 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:38.029400    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-plvzg" event={"ID":"26f57ed5-a830-4098-a2ae-651e7cd00567","Type":"ContainerStarted","Data":"00d51b23e43936957dd8dc6b12b795da2f33f19ae8a3af25f42cf58ac84d8338"}
Sep 10 17:13:38.030687 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:38.030660    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nsv5g" event={"ID":"d29a5496-59b7-42ab-8f4c-b3dbbf771a27","Type":"ContainerStarted","Data":"0c4a8fb4dc39fc44301cfbc3792e52e29ff96c769aacd72c70e8285f03503714"}
Sep 10 17:13:38.032902 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:38.032872    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn" event={"ID":"85265614-4740-49c7-aa44-934ee4ea6fde","Type":"ContainerStarted","Data":"6dc94c5bea25618d45cf295a63a0cced340210d3f6986b37de68a7fca0824ebc"}
Sep 10 17:13:38.034880 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:38.034849    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mn5ts" event={"ID":"47655ffd-db4a-4e92-a4f8-d170c8681d8c","Type":"ContainerStarted","Data":"f2543c6e35179b0bc59dcb18f151e5c0bfb2ae833b457ee8355f6a4e20462998"}
Sep 10 17:13:38.043612 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:38.043552    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-123.ec2.internal" podStartSLOduration=21.043531218 podStartE2EDuration="21.043531218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:13:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:13:22.013736638 +0000 UTC m=+5.694909582" watchObservedRunningTime="2026-09-10 17:13:38.043531218 +0000 UTC m=+21.724704191"
Sep 10 17:13:38.044093 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:38.044028    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gmg9v" podStartSLOduration=4.184981131 podStartE2EDuration="22.044015641s" totalImagesPullingTime="17.85903451s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:13:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:13:38.043399631 +0000 UTC m=+21.724572574" watchObservedRunningTime="2026-09-10 17:13:38.044015641 +0000 UTC m=+21.725188586"
Sep 10 17:13:38.057263 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:38.057217    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mn5ts" podStartSLOduration=3.250509908 podStartE2EDuration="21.05720492s" totalImagesPullingTime="17.806695012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:13:38.057050532 +0000 UTC m=+21.738223519" watchObservedRunningTime="2026-09-10 17:13:38.05720492 +0000 UTC m=+21.738377875"
Sep 10 17:13:38.073564 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:38.073504    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nsv5g" podStartSLOduration=3.22570667 podStartE2EDuration="21.073484384s" totalImagesPullingTime="17.847777714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:13:38.073458541 +0000 UTC m=+21.754631516" watchObservedRunningTime="2026-09-10 17:13:38.073484384 +0000 UTC m=+21.754657344"
Sep 10 17:13:38.087648 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:38.087569    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-plvzg" podStartSLOduration=4.249330451 podStartE2EDuration="22.08754349s" totalImagesPullingTime="17.838213039s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:13:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:13:38.086517363 +0000 UTC m=+21.767690305" watchObservedRunningTime="2026-09-10 17:13:38.08754349 +0000 UTC m=+21.768716449"
Sep 10 17:13:38.828100 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:38.827647    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:38.828100 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:38.828004    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqw5f" podUID="5d278a9b-4d67-4eaf-a7a4-d2e980c2a775"
Sep 10 17:13:38.828100 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:38.827647    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:38.828100 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:38.827649    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:38.828100 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:38.828102    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4rtn" podUID="d993b686-b64a-4643-b53e-2021d0e476ee"
Sep 10 17:13:38.828529 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:38.828267    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzqqc" podUID="7a5b4316-2391-4e12-9a80-12ace78ee49d"
Sep 10 17:13:38.896942 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:38.896910    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 17:13:39.038440 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:39.038358    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn" event={"ID":"85265614-4740-49c7-aa44-934ee4ea6fde","Type":"ContainerStarted","Data":"a8d5b1d14491b69cfdfb84b5f82dad797b59241ecb60934c151b20ac4d68e308"}
Sep 10 17:13:39.039924 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:39.039899    2566 generic.go:356] "Generic (PLEG): container finished" podID="9efe55c6-4eef-4472-846d-4c3861daefa1" containerID="e71cea210593073476949ab8254b1a11d8b8d37f559e63ed30fb1f35c2990208" exitCode=0
Sep 10 17:13:39.040045 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:39.039970    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tp7gl" event={"ID":"9efe55c6-4eef-4472-846d-4c3861daefa1","Type":"ContainerDied","Data":"e71cea210593073476949ab8254b1a11d8b8d37f559e63ed30fb1f35c2990208"}
Sep 10 17:13:39.041653 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:39.041580    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fl4lm" event={"ID":"69cd67eb-da77-4826-9bc2-a5202871c135","Type":"ContainerStarted","Data":"61e595755f4d29f8611858e8b3aafbb1b1cafced4d69bc53d73eba9604ae05df"}
Sep 10 17:13:39.044982 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:39.044954    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl" event={"ID":"7023d897-e2aa-40ea-be34-840f36db1f8d","Type":"ContainerStarted","Data":"c08708ce409d8c8f0cfe4111c1a58a733c2d62cb957107bae2e3359f3078cc94"}
Sep 10 17:13:39.045161 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:39.045037    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl" event={"ID":"7023d897-e2aa-40ea-be34-840f36db1f8d","Type":"ContainerStarted","Data":"53884313035bfc41f6e246c949911dd5f186b48b165da5bbb57912de83bf33ed"}
Sep 10 17:13:39.045161 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:39.045076    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl" event={"ID":"7023d897-e2aa-40ea-be34-840f36db1f8d","Type":"ContainerStarted","Data":"f5a4691ec547d578469d5317f9b042401ca454a223de955581942f2fe0f1faad"}
Sep 10 17:13:39.045161 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:39.045106    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl" event={"ID":"7023d897-e2aa-40ea-be34-840f36db1f8d","Type":"ContainerStarted","Data":"09a480fe6edec06bcc79c7739b5e22b0063469a11dd166a5fd758dc7bf27f47b"}
Sep 10 17:13:39.045273 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:39.045164    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl" event={"ID":"7023d897-e2aa-40ea-be34-840f36db1f8d","Type":"ContainerStarted","Data":"7616e5700aeae3d4e25de31329c99731a5b1cb8c9e0c2267261bd7fec1ce2697"}
Sep 10 17:13:39.045273 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:39.045213    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl" event={"ID":"7023d897-e2aa-40ea-be34-840f36db1f8d","Type":"ContainerStarted","Data":"82f6a0ed9012791bfa57fdf47dcb69c449e193bd100102cb43b70b5407c363a3"}
Sep 10 17:13:39.077061 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:39.077002    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fl4lm" podStartSLOduration=4.23822626 podStartE2EDuration="22.076986217s" totalImagesPullingTime="17.838759957s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:13:39.076869638 +0000 UTC m=+22.758042594" watchObservedRunningTime="2026-09-10 17:13:39.076986217 +0000 UTC m=+22.758159161"
Sep 10 17:13:39.804115 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:39.803957    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T17:13:38.896935329Z","UUID":"eaf15ed3-8364-4e94-9426-54bfae572ebc","Handler":null,"Name":"","Endpoint":""}
Sep 10 17:13:39.805991 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:39.805965    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 17:13:39.806179 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:39.806004    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 17:13:40.049264 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:40.049224    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qxx2h" event={"ID":"0c9c654e-8e9b-43a5-b3fc-5afc8a0e81eb","Type":"ContainerStarted","Data":"b1a348016f9c191406b376f4a5398c9d87ceeeb1ee93dcc27fc79d7663a667e7"}
Sep 10 17:13:40.065317 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:40.065180    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qxx2h" podStartSLOduration=5.221059388 podStartE2EDuration="23.065159267s" totalImagesPullingTime="17.844099879s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:13:40.065015937 +0000 UTC m=+23.746188881" watchObservedRunningTime="2026-09-10 17:13:40.065159267 +0000 UTC m=+23.746332231"
Sep 10 17:13:40.828035 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:40.827947    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:40.828196 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:40.827947    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:40.828196 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:40.828074    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzqqc" podUID="7a5b4316-2391-4e12-9a80-12ace78ee49d"
Sep 10 17:13:40.828196 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:40.827947    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:40.828196 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:40.828176    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqw5f" podUID="5d278a9b-4d67-4eaf-a7a4-d2e980c2a775"
Sep 10 17:13:40.828358 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:40.828225    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4rtn" podUID="d993b686-b64a-4643-b53e-2021d0e476ee"
Sep 10 17:13:41.056036 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:41.055998    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl" event={"ID":"7023d897-e2aa-40ea-be34-840f36db1f8d","Type":"ContainerStarted","Data":"5469dbbc63492502bdef4350cad6737d794045083a58e3af435d3439d69cdcda"}
Sep 10 17:13:41.058584 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:41.058543    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn" event={"ID":"85265614-4740-49c7-aa44-934ee4ea6fde","Type":"ContainerStarted","Data":"5df4388ebbdec7f7d4d18a7d87c6366fd0b537399adc7f83cc7b906dd1598904"}
Sep 10 17:13:41.077157 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:41.077091    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c26xn" podStartSLOduration=4.188166414 podStartE2EDuration="24.077072292s" totalImagesPullingTime="19.888905878s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:13:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:13:41.076730347 +0000 UTC m=+24.757903292" watchObservedRunningTime="2026-09-10 17:13:41.077072292 +0000 UTC m=+24.758245234"
Sep 10 17:13:42.828162 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:42.828110    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:42.828622 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:42.828110    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:42.828622 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:42.828269    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqw5f" podUID="5d278a9b-4d67-4eaf-a7a4-d2e980c2a775"
Sep 10 17:13:42.828622 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:42.828317    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:42.828622 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:42.828408    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4rtn" podUID="d993b686-b64a-4643-b53e-2021d0e476ee"
Sep 10 17:13:42.828622 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:42.828490    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzqqc" podUID="7a5b4316-2391-4e12-9a80-12ace78ee49d"
Sep 10 17:13:42.983575 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:42.983482    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mn5ts"
Sep 10 17:13:42.984197 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:42.984177    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mn5ts"
Sep 10 17:13:43.065045 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:43.065015    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mn5ts"
Sep 10 17:13:43.065610 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:43.065586    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mn5ts"
Sep 10 17:13:44.070738 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:44.070519    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl" event={"ID":"7023d897-e2aa-40ea-be34-840f36db1f8d","Type":"ContainerStarted","Data":"6cc3bc8f4fc68919032702a2f9dd0b5cd29b007900730887410fe23bec0a0f04"}
Sep 10 17:13:44.071265 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:44.070819    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:44.071265 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:44.070871    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:44.072697 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:44.072663    2566 generic.go:356] "Generic (PLEG): container finished" podID="9efe55c6-4eef-4472-846d-4c3861daefa1" containerID="376399b115a435658acd6def001f699677b554c047a25f8a2f1b1d9d11c19028" exitCode=0
Sep 10 17:13:44.072936 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:44.072730    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tp7gl" event={"ID":"9efe55c6-4eef-4472-846d-4c3861daefa1","Type":"ContainerDied","Data":"376399b115a435658acd6def001f699677b554c047a25f8a2f1b1d9d11c19028"}
Sep 10 17:13:44.086956 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:44.086921    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:44.098015 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:44.097962    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl" podStartSLOduration=9.014650234 podStartE2EDuration="27.09794671s" totalImagesPullingTime="18.083296476s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:13:44.09634487 +0000 UTC m=+27.777517814" watchObservedRunningTime="2026-09-10 17:13:44.09794671 +0000 UTC m=+27.779119653"
Sep 10 17:13:44.828111 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:44.828077    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:44.828401 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:44.828078    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:44.828401 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:44.828225    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqw5f" podUID="5d278a9b-4d67-4eaf-a7a4-d2e980c2a775"
Sep 10 17:13:44.828401 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:44.828089    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:44.828401 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:44.828288    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4rtn" podUID="d993b686-b64a-4643-b53e-2021d0e476ee"
Sep 10 17:13:44.828401 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:44.828382    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzqqc" podUID="7a5b4316-2391-4e12-9a80-12ace78ee49d"
Sep 10 17:13:45.078350 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:45.078258    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tp7gl" event={"ID":"9efe55c6-4eef-4472-846d-4c3861daefa1","Type":"ContainerStarted","Data":"638cdf759054295f9cf6ff2d5f417ee73e8e2d98f87ae7bc627189edf9c58596"}
Sep 10 17:13:45.082148 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:45.082100    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:45.097983 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:45.097956    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:13:45.287517 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:45.287482    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qqw5f"]
Sep 10 17:13:45.287685 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:45.287601    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:45.287748 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:45.287706    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqw5f" podUID="5d278a9b-4d67-4eaf-a7a4-d2e980c2a775"
Sep 10 17:13:45.290209 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:45.290178    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rzqqc"]
Sep 10 17:13:45.290355 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:45.290315    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:45.290436 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:45.290417    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzqqc" podUID="7a5b4316-2391-4e12-9a80-12ace78ee49d"
Sep 10 17:13:45.293902 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:45.293867    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l4rtn"]
Sep 10 17:13:45.294034 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:45.293999    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:45.294119 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:45.294101    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4rtn" podUID="d993b686-b64a-4643-b53e-2021d0e476ee"
Sep 10 17:13:46.082696 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:46.082659    2566 generic.go:356] "Generic (PLEG): container finished" podID="9efe55c6-4eef-4472-846d-4c3861daefa1" containerID="638cdf759054295f9cf6ff2d5f417ee73e8e2d98f87ae7bc627189edf9c58596" exitCode=0
Sep 10 17:13:46.083118 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:46.082734    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tp7gl" event={"ID":"9efe55c6-4eef-4472-846d-4c3861daefa1","Type":"ContainerDied","Data":"638cdf759054295f9cf6ff2d5f417ee73e8e2d98f87ae7bc627189edf9c58596"}
Sep 10 17:13:46.828448 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:46.828417    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:46.828598 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:46.828513    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:46.828598 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:46.828527    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4rtn" podUID="d993b686-b64a-4643-b53e-2021d0e476ee"
Sep 10 17:13:46.828598 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:46.828576    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:46.828753 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:46.828677    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqw5f" podUID="5d278a9b-4d67-4eaf-a7a4-d2e980c2a775"
Sep 10 17:13:46.828806 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:46.828778    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzqqc" podUID="7a5b4316-2391-4e12-9a80-12ace78ee49d"
Sep 10 17:13:46.838425 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:46.836916    2566 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9efe55c6_4eef_4472_846d_4c3861daefa1.slice/crio-137a1c992628fae713d57b0bd7632b62832b53a48e6daf9113e14d62bd0286ab.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 17:13:47.088490 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:47.088448    2566 generic.go:356] "Generic (PLEG): container finished" podID="9efe55c6-4eef-4472-846d-4c3861daefa1" containerID="137a1c992628fae713d57b0bd7632b62832b53a48e6daf9113e14d62bd0286ab" exitCode=0
Sep 10 17:13:47.089174 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:47.088525    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tp7gl" event={"ID":"9efe55c6-4eef-4472-846d-4c3861daefa1","Type":"ContainerDied","Data":"137a1c992628fae713d57b0bd7632b62832b53a48e6daf9113e14d62bd0286ab"}
Sep 10 17:13:48.828534 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:48.828302    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:48.828968 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:48.828302    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:48.828968 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:48.828637    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4rtn" podUID="d993b686-b64a-4643-b53e-2021d0e476ee"
Sep 10 17:13:48.828968 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:48.828714    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqw5f" podUID="5d278a9b-4d67-4eaf-a7a4-d2e980c2a775"
Sep 10 17:13:48.828968 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:48.828301    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:48.828968 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:48.828821    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzqqc" podUID="7a5b4316-2391-4e12-9a80-12ace78ee49d"
Sep 10 17:13:50.488935 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.488895    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs\") pod \"network-metrics-daemon-rzqqc\" (UID: \"7a5b4316-2391-4e12-9a80-12ace78ee49d\") " pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:50.489433 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:50.489062    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:13:50.489433 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:50.489157    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs podName:7a5b4316-2391-4e12-9a80-12ace78ee49d nodeName:}" failed. No retries permitted until 2026-09-10 17:14:22.489118352 +0000 UTC m=+66.170291274 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs") pod "network-metrics-daemon-rzqqc" (UID: "7a5b4316-2391-4e12-9a80-12ace78ee49d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:13:50.589303 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.589263    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plszb\" (UniqueName: \"kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb\") pod \"network-check-target-l4rtn\" (UID: \"d993b686-b64a-4643-b53e-2021d0e476ee\") " pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:50.589505 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.589331    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret\") pod \"global-pull-secret-syncer-qqw5f\" (UID: \"5d278a9b-4d67-4eaf-a7a4-d2e980c2a775\") " pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:50.589505 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:50.589454    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:13:50.589505 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:50.589473    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:13:50.589505 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:50.589505    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:13:50.589684 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:50.589532    2566 projected.go:196] Error preparing data for projected volume kube-api-access-plszb for pod openshift-network-diagnostics/network-check-target-l4rtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:13:50.589684 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:50.589512    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret podName:5d278a9b-4d67-4eaf-a7a4-d2e980c2a775 nodeName:}" failed. No retries permitted until 2026-09-10 17:14:22.589498374 +0000 UTC m=+66.270671296 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret") pod "global-pull-secret-syncer-qqw5f" (UID: "5d278a9b-4d67-4eaf-a7a4-d2e980c2a775") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:13:50.589684 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:50.589596    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb podName:d993b686-b64a-4643-b53e-2021d0e476ee nodeName:}" failed. No retries permitted until 2026-09-10 17:14:22.589578207 +0000 UTC m=+66.270751131 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-plszb" (UniqueName: "kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb") pod "network-check-target-l4rtn" (UID: "d993b686-b64a-4643-b53e-2021d0e476ee") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:13:50.624206 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.624115    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-123.ec2.internal" event="NodeReady"
Sep 10 17:13:50.624366 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.624319    2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 17:13:50.661522 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.661474    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8467c67c97-9dd9f"]
Sep 10 17:13:50.697956 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.697921    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8467c67c97-9dd9f"]
Sep 10 17:13:50.698204 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.697970    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pwhzb"]
Sep 10 17:13:50.698204 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.698063    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.701327 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.701294    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 17:13:50.701504 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.701334    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 17:13:50.701504 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.701374    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 17:13:50.701675 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.701658    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gj2cb\""
Sep 10 17:13:50.710084 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.710059    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 17:13:50.711820 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.711796    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vkbjg"]
Sep 10 17:13:50.711971 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.711948    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:13:50.714458 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.714435    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2rm4x\""
Sep 10 17:13:50.714579 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.714435    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 17:13:50.714579 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.714488    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 17:13:50.728007 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.727979    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pwhzb"]
Sep 10 17:13:50.728195 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.728039    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vkbjg"
Sep 10 17:13:50.728195 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.728055    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vkbjg"]
Sep 10 17:13:50.730766 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.730735    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 17:13:50.730924 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.730742    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 17:13:50.730924 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.730920    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nn99k\""
Sep 10 17:13:50.731299 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.731270    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 17:13:50.790325 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.790286    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cx9l\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-kube-api-access-4cx9l\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.790528 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.790338    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxqqg\" (UniqueName: \"kubernetes.io/projected/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-kube-api-access-gxqqg\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:13:50.790528 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.790377    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24d26347-fb61-4ce9-97dd-61c6c634582a-image-registry-private-configuration\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.790528 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.790403    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-certificates\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.790528 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.790437    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7cf62bc7-5bc1-4f6a-b73b-a7e488ab8c6f-cert\") pod \"ingress-canary-vkbjg\" (UID: \"7cf62bc7-5bc1-4f6a-b73b-a7e488ab8c6f\") " pod="openshift-ingress-canary/ingress-canary-vkbjg"
Sep 10 17:13:50.790528 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.790496    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24d26347-fb61-4ce9-97dd-61c6c634582a-installation-pull-secrets\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.790713 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.790528    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-config-volume\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:13:50.790713 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.790564    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:13:50.790713 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.790578    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-tmp-dir\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:13:50.790713 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.790638    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h75rl\" (UniqueName: \"kubernetes.io/projected/7cf62bc7-5bc1-4f6a-b73b-a7e488ab8c6f-kube-api-access-h75rl\") pod \"ingress-canary-vkbjg\" (UID: \"7cf62bc7-5bc1-4f6a-b73b-a7e488ab8c6f\") " pod="openshift-ingress-canary/ingress-canary-vkbjg"
Sep 10 17:13:50.790713 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.790698    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24d26347-fb61-4ce9-97dd-61c6c634582a-ca-trust-extracted\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.790944 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.790722    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24d26347-fb61-4ce9-97dd-61c6c634582a-trusted-ca\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.790944 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.790807    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-bound-sa-token\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.790944 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.790850    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.828086 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.828040    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:13:50.828305 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.828097    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:13:50.828305 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.828057    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:13:50.831045 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.831012    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g7xkh\""
Sep 10 17:13:50.831246 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.831013    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 17:13:50.831246 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.831085    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 17:13:50.831246 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.831092    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n4rqr\""
Sep 10 17:13:50.831246 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.831156    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 17:13:50.831246 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.831015    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 17:13:50.891977 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.891886    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24d26347-fb61-4ce9-97dd-61c6c634582a-installation-pull-secrets\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.891977 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.891922    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-config-volume\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:13:50.891977 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.891961    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:13:50.892324 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.891990    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-tmp-dir\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:13:50.892324 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.892036    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h75rl\" (UniqueName: \"kubernetes.io/projected/7cf62bc7-5bc1-4f6a-b73b-a7e488ab8c6f-kube-api-access-h75rl\") pod \"ingress-canary-vkbjg\" (UID: \"7cf62bc7-5bc1-4f6a-b73b-a7e488ab8c6f\") " pod="openshift-ingress-canary/ingress-canary-vkbjg"
Sep 10 17:13:50.892324 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.892077    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24d26347-fb61-4ce9-97dd-61c6c634582a-ca-trust-extracted\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.892324 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.892091    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24d26347-fb61-4ce9-97dd-61c6c634582a-trusted-ca\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.892324 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.892115    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-bound-sa-token\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.892324 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.892157    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.892324 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.892204    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cx9l\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-kube-api-access-4cx9l\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.892324 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.892225    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxqqg\" (UniqueName: \"kubernetes.io/projected/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-kube-api-access-gxqqg\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:13:50.892324 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.892251    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24d26347-fb61-4ce9-97dd-61c6c634582a-image-registry-private-configuration\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.892324 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.892278    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-certificates\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.892324 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.892311    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7cf62bc7-5bc1-4f6a-b73b-a7e488ab8c6f-cert\") pod \"ingress-canary-vkbjg\" (UID: \"7cf62bc7-5bc1-4f6a-b73b-a7e488ab8c6f\") " pod="openshift-ingress-canary/ingress-canary-vkbjg"
Sep 10 17:13:50.892850 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.892624    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-config-volume\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:13:50.893116 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:50.892975    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 17:13:50.893116 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:50.893038    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls podName:2f11bf9c-45a3-4b27-8f8d-4a1d52999da2 nodeName:}" failed. No retries permitted until 2026-09-10 17:13:51.393020667 +0000 UTC m=+35.074193602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls") pod "dns-default-pwhzb" (UID: "2f11bf9c-45a3-4b27-8f8d-4a1d52999da2") : secret "dns-default-metrics-tls" not found
Sep 10 17:13:50.893450 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.893427    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-tmp-dir\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:13:50.893506 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.893446    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24d26347-fb61-4ce9-97dd-61c6c634582a-ca-trust-extracted\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.893506 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:50.892982    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 17:13:50.893506 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:50.893485    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8467c67c97-9dd9f: secret "image-registry-tls" not found
Sep 10 17:13:50.893608 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:50.893541    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls podName:24d26347-fb61-4ce9-97dd-61c6c634582a nodeName:}" failed. No retries permitted until 2026-09-10 17:13:51.393522639 +0000 UTC m=+35.074695574 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls") pod "image-registry-8467c67c97-9dd9f" (UID: "24d26347-fb61-4ce9-97dd-61c6c634582a") : secret "image-registry-tls" not found
Sep 10 17:13:50.894069 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.894042    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-certificates\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.894679 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.894589    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24d26347-fb61-4ce9-97dd-61c6c634582a-trusted-ca\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.897549 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.897523    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24d26347-fb61-4ce9-97dd-61c6c634582a-installation-pull-secrets\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.897688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.897524    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24d26347-fb61-4ce9-97dd-61c6c634582a-image-registry-private-configuration\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.897688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.897583    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7cf62bc7-5bc1-4f6a-b73b-a7e488ab8c6f-cert\") pod \"ingress-canary-vkbjg\" (UID: \"7cf62bc7-5bc1-4f6a-b73b-a7e488ab8c6f\") " pod="openshift-ingress-canary/ingress-canary-vkbjg"
Sep 10 17:13:50.910433 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.910404    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h75rl\" (UniqueName: \"kubernetes.io/projected/7cf62bc7-5bc1-4f6a-b73b-a7e488ab8c6f-kube-api-access-h75rl\") pod \"ingress-canary-vkbjg\" (UID: \"7cf62bc7-5bc1-4f6a-b73b-a7e488ab8c6f\") " pod="openshift-ingress-canary/ingress-canary-vkbjg"
Sep 10 17:13:50.910809 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.910782    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-bound-sa-token\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:50.910912 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.910788    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxqqg\" (UniqueName: \"kubernetes.io/projected/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-kube-api-access-gxqqg\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:13:50.911440 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:50.911415    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cx9l\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-kube-api-access-4cx9l\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:51.038359 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:51.038315    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vkbjg"
Sep 10 17:13:51.395215 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:51.395169    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:13:51.395397 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:51.395256    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:51.395397 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:51.395348    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 17:13:51.395397 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:51.395370    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 17:13:51.395397 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:51.395386    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8467c67c97-9dd9f: secret "image-registry-tls" not found
Sep 10 17:13:51.395535 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:51.395437    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls podName:24d26347-fb61-4ce9-97dd-61c6c634582a nodeName:}" failed. No retries permitted until 2026-09-10 17:13:52.395415281 +0000 UTC m=+36.076588205 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls") pod "image-registry-8467c67c97-9dd9f" (UID: "24d26347-fb61-4ce9-97dd-61c6c634582a") : secret "image-registry-tls" not found
Sep 10 17:13:51.395535 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:51.395453    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls podName:2f11bf9c-45a3-4b27-8f8d-4a1d52999da2 nodeName:}" failed. No retries permitted until 2026-09-10 17:13:52.395444945 +0000 UTC m=+36.076617866 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls") pod "dns-default-pwhzb" (UID: "2f11bf9c-45a3-4b27-8f8d-4a1d52999da2") : secret "dns-default-metrics-tls" not found
Sep 10 17:13:51.506016 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:51.505983    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vkbjg"]
Sep 10 17:13:51.511222 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:13:51.511190    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cf62bc7_5bc1_4f6a_b73b_a7e488ab8c6f.slice/crio-378267df9ef35a4fc65e7592175e5a5911b45d1382060b933c9141d0d7dae115 WatchSource:0}: Error finding container 378267df9ef35a4fc65e7592175e5a5911b45d1382060b933c9141d0d7dae115: Status 404 returned error can't find the container with id 378267df9ef35a4fc65e7592175e5a5911b45d1382060b933c9141d0d7dae115
Sep 10 17:13:52.104875 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:52.104835    2566 generic.go:356] "Generic (PLEG): container finished" podID="9efe55c6-4eef-4472-846d-4c3861daefa1" containerID="b788d5f7c9156d375d6f38208891da043343a48bbcafe18bbe2b67be7fcaa25d" exitCode=0
Sep 10 17:13:52.105060 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:52.104919    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tp7gl" event={"ID":"9efe55c6-4eef-4472-846d-4c3861daefa1","Type":"ContainerDied","Data":"b788d5f7c9156d375d6f38208891da043343a48bbcafe18bbe2b67be7fcaa25d"}
Sep 10 17:13:52.106116 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:52.106084    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vkbjg" event={"ID":"7cf62bc7-5bc1-4f6a-b73b-a7e488ab8c6f","Type":"ContainerStarted","Data":"378267df9ef35a4fc65e7592175e5a5911b45d1382060b933c9141d0d7dae115"}
Sep 10 17:13:52.403985 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:52.403892    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:13:52.403985 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:52.403971    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:52.404254 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:52.404075    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 17:13:52.404254 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:52.404084    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 17:13:52.404254 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:52.404186    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8467c67c97-9dd9f: secret "image-registry-tls" not found
Sep 10 17:13:52.404254 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:52.404166    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls podName:2f11bf9c-45a3-4b27-8f8d-4a1d52999da2 nodeName:}" failed. No retries permitted until 2026-09-10 17:13:54.404146658 +0000 UTC m=+38.085319596 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls") pod "dns-default-pwhzb" (UID: "2f11bf9c-45a3-4b27-8f8d-4a1d52999da2") : secret "dns-default-metrics-tls" not found
Sep 10 17:13:52.404439 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:52.404266    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls podName:24d26347-fb61-4ce9-97dd-61c6c634582a nodeName:}" failed. No retries permitted until 2026-09-10 17:13:54.404246031 +0000 UTC m=+38.085418960 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls") pod "image-registry-8467c67c97-9dd9f" (UID: "24d26347-fb61-4ce9-97dd-61c6c634582a") : secret "image-registry-tls" not found
Sep 10 17:13:53.113378 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:53.113293    2566 generic.go:356] "Generic (PLEG): container finished" podID="9efe55c6-4eef-4472-846d-4c3861daefa1" containerID="0804216489bbb5277a818f9d2d8de14cd2af00c0feafd9552f8bb9adb5160035" exitCode=0
Sep 10 17:13:53.113378 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:53.113333    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tp7gl" event={"ID":"9efe55c6-4eef-4472-846d-4c3861daefa1","Type":"ContainerDied","Data":"0804216489bbb5277a818f9d2d8de14cd2af00c0feafd9552f8bb9adb5160035"}
Sep 10 17:13:53.328322 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:53.328275    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:13:53.424420 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:53.424327    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:13:53.487553 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:53.487515    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:13:54.118982 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:54.118941    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vkbjg" event={"ID":"7cf62bc7-5bc1-4f6a-b73b-a7e488ab8c6f","Type":"ContainerStarted","Data":"39c795727b434fbd475bb334d944564d8d077ff31a8040a0b75916c53e1542c9"}
Sep 10 17:13:54.122777 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:54.122748    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tp7gl" event={"ID":"9efe55c6-4eef-4472-846d-4c3861daefa1","Type":"ContainerStarted","Data":"4a62854a1bfa9853adb199a50746629ae78dded5b95c1b0c4a47860340f902fd"}
Sep 10 17:13:54.135574 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:54.135507    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vkbjg" podStartSLOduration=1.688918352 podStartE2EDuration="4.135486821s" totalImagesPullingTime="2.446568469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:13:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:13:54.134427555 +0000 UTC m=+37.815600498" watchObservedRunningTime="2026-09-10 17:13:54.135486821 +0000 UTC m=+37.816659763"
Sep 10 17:13:54.159855 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:54.159803    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tp7gl" podStartSLOduration=10.138872186 podStartE2EDuration="38.15978699s" totalImagesPullingTime="28.020914804s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:13:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:13:54.15919922 +0000 UTC m=+37.840372163" watchObservedRunningTime="2026-09-10 17:13:54.15978699 +0000 UTC m=+37.840959932"
Sep 10 17:13:54.288664 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:54.288627    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:13:54.365120 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:54.365033    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:13:54.422070 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:54.422040    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:13:54.422275 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:54.422103    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:54.422275 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:54.422229    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 17:13:54.422275 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:54.422246    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8467c67c97-9dd9f: secret "image-registry-tls" not found
Sep 10 17:13:54.422275 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:54.422230    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 17:13:54.422405 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:54.422301    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls podName:24d26347-fb61-4ce9-97dd-61c6c634582a nodeName:}" failed. No retries permitted until 2026-09-10 17:13:58.422285844 +0000 UTC m=+42.103458764 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls") pod "image-registry-8467c67c97-9dd9f" (UID: "24d26347-fb61-4ce9-97dd-61c6c634582a") : secret "image-registry-tls" not found
Sep 10 17:13:54.422405 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:54.422324    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls podName:2f11bf9c-45a3-4b27-8f8d-4a1d52999da2 nodeName:}" failed. No retries permitted until 2026-09-10 17:13:58.422306093 +0000 UTC m=+42.103479025 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls") pod "dns-default-pwhzb" (UID: "2f11bf9c-45a3-4b27-8f8d-4a1d52999da2") : secret "dns-default-metrics-tls" not found
Sep 10 17:13:56.182078 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:56.182044    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:13:56.254811 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:56.254776    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:13:57.864330 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:57.864297    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:13:57.965387 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:57.965355    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:13:58.451688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:58.451642    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:13:58.451875 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:13:58.451701    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:13:58.451875 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:58.451800    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 17:13:58.451875 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:58.451824    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8467c67c97-9dd9f: secret "image-registry-tls" not found
Sep 10 17:13:58.451875 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:58.451826    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 17:13:58.451875 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:58.451877    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls podName:2f11bf9c-45a3-4b27-8f8d-4a1d52999da2 nodeName:}" failed. No retries permitted until 2026-09-10 17:14:06.451863838 +0000 UTC m=+50.133036759 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls") pod "dns-default-pwhzb" (UID: "2f11bf9c-45a3-4b27-8f8d-4a1d52999da2") : secret "dns-default-metrics-tls" not found
Sep 10 17:13:58.452034 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:13:58.451891    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls podName:24d26347-fb61-4ce9-97dd-61c6c634582a nodeName:}" failed. No retries permitted until 2026-09-10 17:14:06.45188501 +0000 UTC m=+50.133057931 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls") pod "image-registry-8467c67c97-9dd9f" (UID: "24d26347-fb61-4ce9-97dd-61c6c634582a") : secret "image-registry-tls" not found
Sep 10 17:14:00.307849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:00.307803    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:00.361786 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:00.361753    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:01.616966 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:01.616930    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:03.096729 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:03.096673    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:04.210638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:04.210603    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:04.273523 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:04.273478    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:06.512096 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:06.512054    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:14:06.512521 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:06.512116    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:14:06.512521 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:06.512230    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 17:14:06.512521 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:06.512296    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls podName:2f11bf9c-45a3-4b27-8f8d-4a1d52999da2 nodeName:}" failed. No retries permitted until 2026-09-10 17:14:22.512279383 +0000 UTC m=+66.193452309 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls") pod "dns-default-pwhzb" (UID: "2f11bf9c-45a3-4b27-8f8d-4a1d52999da2") : secret "dns-default-metrics-tls" not found
Sep 10 17:14:06.512521 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:06.512233    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 17:14:06.512521 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:06.512334    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8467c67c97-9dd9f: secret "image-registry-tls" not found
Sep 10 17:14:06.512521 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:06.512382    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls podName:24d26347-fb61-4ce9-97dd-61c6c634582a nodeName:}" failed. No retries permitted until 2026-09-10 17:14:22.512367306 +0000 UTC m=+66.193540256 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls") pod "image-registry-8467c67c97-9dd9f" (UID: "24d26347-fb61-4ce9-97dd-61c6c634582a") : secret "image-registry-tls" not found
Sep 10 17:14:17.100417 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:17.100386    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ssbcl"
Sep 10 17:14:17.255663 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:17.255630    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:17.327424 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:17.327370    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:20.874347 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:20.874310    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:20.942035 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:20.942000    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:22.529568 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:22.529523    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:14:22.530048 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:22.529574    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs\") pod \"network-metrics-daemon-rzqqc\" (UID: \"7a5b4316-2391-4e12-9a80-12ace78ee49d\") " pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:14:22.530048 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:22.529632    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:14:22.530048 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:22.529696    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 17:14:22.530048 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:22.529721    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 17:14:22.530048 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:22.529733    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8467c67c97-9dd9f: secret "image-registry-tls" not found
Sep 10 17:14:22.530048 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:22.529782    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls podName:2f11bf9c-45a3-4b27-8f8d-4a1d52999da2 nodeName:}" failed. No retries permitted until 2026-09-10 17:14:54.529760523 +0000 UTC m=+98.210933463 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls") pod "dns-default-pwhzb" (UID: "2f11bf9c-45a3-4b27-8f8d-4a1d52999da2") : secret "dns-default-metrics-tls" not found
Sep 10 17:14:22.530048 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:22.529796    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls podName:24d26347-fb61-4ce9-97dd-61c6c634582a nodeName:}" failed. No retries permitted until 2026-09-10 17:14:54.529790734 +0000 UTC m=+98.210963658 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls") pod "image-registry-8467c67c97-9dd9f" (UID: "24d26347-fb61-4ce9-97dd-61c6c634582a") : secret "image-registry-tls" not found
Sep 10 17:14:22.532409 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:22.532379    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 17:14:22.540764 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:22.540737    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 17:14:22.540872 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:22.540810    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs podName:7a5b4316-2391-4e12-9a80-12ace78ee49d nodeName:}" failed. No retries permitted until 2026-09-10 17:15:26.540791227 +0000 UTC m=+130.221964148 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs") pod "network-metrics-daemon-rzqqc" (UID: "7a5b4316-2391-4e12-9a80-12ace78ee49d") : secret "metrics-daemon-secret" not found
Sep 10 17:14:22.630808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:22.630754    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plszb\" (UniqueName: \"kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb\") pod \"network-check-target-l4rtn\" (UID: \"d993b686-b64a-4643-b53e-2021d0e476ee\") " pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:14:22.630979 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:22.630843    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret\") pod \"global-pull-secret-syncer-qqw5f\" (UID: \"5d278a9b-4d67-4eaf-a7a4-d2e980c2a775\") " pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:14:22.633708 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:22.633680    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 17:14:22.633850 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:22.633782    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 17:14:22.644098 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:22.644076    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 17:14:22.645211 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:22.645188    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d278a9b-4d67-4eaf-a7a4-d2e980c2a775-original-pull-secret\") pod \"global-pull-secret-syncer-qqw5f\" (UID: \"5d278a9b-4d67-4eaf-a7a4-d2e980c2a775\") " pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:14:22.647059 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:22.647042    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqw5f"
Sep 10 17:14:22.654669 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:22.654645    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plszb\" (UniqueName: \"kubernetes.io/projected/d993b686-b64a-4643-b53e-2021d0e476ee-kube-api-access-plszb\") pod \"network-check-target-l4rtn\" (UID: \"d993b686-b64a-4643-b53e-2021d0e476ee\") " pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:14:22.918391 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:22.918298    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qqw5f"]
Sep 10 17:14:22.921991 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:14:22.921949    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d278a9b_4d67_4eaf_a7a4_d2e980c2a775.slice/crio-c49047f8cad2bf7d759f673ad6c30d3ba602ba048afd832e536b16c23ddb5644 WatchSource:0}: Error finding container c49047f8cad2bf7d759f673ad6c30d3ba602ba048afd832e536b16c23ddb5644: Status 404 returned error can't find the container with id c49047f8cad2bf7d759f673ad6c30d3ba602ba048afd832e536b16c23ddb5644
Sep 10 17:14:22.942611 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:22.942574    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g7xkh\""
Sep 10 17:14:22.950587 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:22.950562    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:14:23.195495 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:23.195410    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qqw5f" event={"ID":"5d278a9b-4d67-4eaf-a7a4-d2e980c2a775","Type":"ContainerStarted","Data":"c49047f8cad2bf7d759f673ad6c30d3ba602ba048afd832e536b16c23ddb5644"}
Sep 10 17:14:23.219594 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:23.219552    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l4rtn"]
Sep 10 17:14:23.223172 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:14:23.223142    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd993b686_b64a_4643_b53e_2021d0e476ee.slice/crio-e81fc2c1b6be07797da7f59d71b003f0170bf85381ce190a2a4d2971c78e5ad7 WatchSource:0}: Error finding container e81fc2c1b6be07797da7f59d71b003f0170bf85381ce190a2a4d2971c78e5ad7: Status 404 returned error can't find the container with id e81fc2c1b6be07797da7f59d71b003f0170bf85381ce190a2a4d2971c78e5ad7
Sep 10 17:14:24.200683 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:24.200642    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l4rtn" event={"ID":"d993b686-b64a-4643-b53e-2021d0e476ee","Type":"ContainerStarted","Data":"e81fc2c1b6be07797da7f59d71b003f0170bf85381ce190a2a4d2971c78e5ad7"}
Sep 10 17:14:24.566435 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:24.566380    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:24.628950 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:24.628909    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:28.212562 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:28.212523    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l4rtn" event={"ID":"d993b686-b64a-4643-b53e-2021d0e476ee","Type":"ContainerStarted","Data":"59596c8bb8242dfb27c5bb50c6f3af062f6b5ac1ff7f76aa7f69baa8998962ad"}
Sep 10 17:14:28.213088 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:28.212590    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:14:28.214069 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:28.214047    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qqw5f" event={"ID":"5d278a9b-4d67-4eaf-a7a4-d2e980c2a775","Type":"ContainerStarted","Data":"aa61164a1df3c20116d56448f017331f09a933865a9966faa20be99ae8970283"}
Sep 10 17:14:28.228885 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:28.228819    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l4rtn" podStartSLOduration=66.927637493 podStartE2EDuration="1m11.228795818s" totalImagesPullingTime="4.301158325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:14:28.228545142 +0000 UTC m=+71.909718086" watchObservedRunningTime="2026-09-10 17:14:28.228795818 +0000 UTC m=+71.909968762"
Sep 10 17:14:28.244377 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:28.244326    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qqw5f" podStartSLOduration=66.637596582 podStartE2EDuration="1m11.244312706s" totalImagesPullingTime="4.606716124s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:14:28.243726339 +0000 UTC m=+71.924899294" watchObservedRunningTime="2026-09-10 17:14:28.244312706 +0000 UTC m=+71.925485648"
Sep 10 17:14:30.858948 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:30.858905    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:30.961519 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:30.961483    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:40.327242 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:40.327082    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:41.164811 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:41.164773    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-pvqls"]
Sep 10 17:14:41.169089 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:41.169061    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-pvqls"
Sep 10 17:14:41.171633 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:41.171605    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 10 17:14:41.171855 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:41.171839    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 10 17:14:41.172570 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:41.172556    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-bbvm2\""
Sep 10 17:14:41.181878 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:41.181847    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-pvqls"]
Sep 10 17:14:41.265793 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:41.265752    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/367ec36e-c951-4ec0-9ddc-08af4f0f6714-nginx-conf\") pod \"networking-console-plugin-79977f9844-pvqls\" (UID: \"367ec36e-c951-4ec0-9ddc-08af4f0f6714\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pvqls"
Sep 10 17:14:41.265793 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:41.265798    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pvqls\" (UID: \"367ec36e-c951-4ec0-9ddc-08af4f0f6714\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pvqls"
Sep 10 17:14:41.356169 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:41.356119    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:41.367201 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:41.367163    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/367ec36e-c951-4ec0-9ddc-08af4f0f6714-nginx-conf\") pod \"networking-console-plugin-79977f9844-pvqls\" (UID: \"367ec36e-c951-4ec0-9ddc-08af4f0f6714\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pvqls"
Sep 10 17:14:41.367426 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:41.367220    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pvqls\" (UID: \"367ec36e-c951-4ec0-9ddc-08af4f0f6714\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pvqls"
Sep 10 17:14:41.367426 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:41.367363    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 17:14:41.367538 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:41.367452    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert podName:367ec36e-c951-4ec0-9ddc-08af4f0f6714 nodeName:}" failed. No retries permitted until 2026-09-10 17:14:41.867430127 +0000 UTC m=+85.548603066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-pvqls" (UID: "367ec36e-c951-4ec0-9ddc-08af4f0f6714") : secret "networking-console-plugin-cert" not found
Sep 10 17:14:41.367819 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:41.367795    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/367ec36e-c951-4ec0-9ddc-08af4f0f6714-nginx-conf\") pod \"networking-console-plugin-79977f9844-pvqls\" (UID: \"367ec36e-c951-4ec0-9ddc-08af4f0f6714\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pvqls"
Sep 10 17:14:41.424031 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:41.423944    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:41.870621 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:41.870575    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pvqls\" (UID: \"367ec36e-c951-4ec0-9ddc-08af4f0f6714\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pvqls"
Sep 10 17:14:41.870862 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:41.870750    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 17:14:41.870862 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:41.870845    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert podName:367ec36e-c951-4ec0-9ddc-08af4f0f6714 nodeName:}" failed. No retries permitted until 2026-09-10 17:14:42.87082571 +0000 UTC m=+86.551998631 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-pvqls" (UID: "367ec36e-c951-4ec0-9ddc-08af4f0f6714") : secret "networking-console-plugin-cert" not found
Sep 10 17:14:42.876857 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:42.876817    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pvqls\" (UID: \"367ec36e-c951-4ec0-9ddc-08af4f0f6714\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pvqls"
Sep 10 17:14:42.877310 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:42.876969    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 17:14:42.877310 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:42.877035    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert podName:367ec36e-c951-4ec0-9ddc-08af4f0f6714 nodeName:}" failed. No retries permitted until 2026-09-10 17:14:44.87701983 +0000 UTC m=+88.558192755 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-pvqls" (UID: "367ec36e-c951-4ec0-9ddc-08af4f0f6714") : secret "networking-console-plugin-cert" not found
Sep 10 17:14:43.704732 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:43.704691    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:44.831272 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:44.831238    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:44.890559 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:44.890518    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pvqls\" (UID: \"367ec36e-c951-4ec0-9ddc-08af4f0f6714\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pvqls"
Sep 10 17:14:44.890742 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:44.890681    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 17:14:44.890793 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:44.890753    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert podName:367ec36e-c951-4ec0-9ddc-08af4f0f6714 nodeName:}" failed. No retries permitted until 2026-09-10 17:14:48.890736218 +0000 UTC m=+92.571909140 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-pvqls" (UID: "367ec36e-c951-4ec0-9ddc-08af4f0f6714") : secret "networking-console-plugin-cert" not found
Sep 10 17:14:44.923364 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:44.923333    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:45.023757 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:45.023720    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:45.998654 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:45.998616    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:46.011669 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.011632    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6n9rh"]
Sep 10 17:14:46.014884 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.014858    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:46.017441 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.017415    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 17:14:46.018112 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.018094    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2hvq7\""
Sep 10 17:14:46.018184 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.018110    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 17:14:46.018184 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.018099    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 17:14:46.018184 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.018149    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 17:14:46.035708 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.035675    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6n9rh"]
Sep 10 17:14:46.087709 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.087678    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:46.101779 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.101744    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:46.101917 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.101796    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5dtq\" (UniqueName: \"kubernetes.io/projected/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-api-access-w5dtq\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:46.101917 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.101890    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-crio-socket\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:46.102005 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.101917    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-data-volume\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:46.102005 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.101939    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:46.202915 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.202880    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-crio-socket\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:46.202915 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.202916    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-data-volume\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:46.203062 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.202938    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:46.203062 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:46.203001    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 17:14:46.203062 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.203017    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-crio-socket\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:46.203195 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:46.203073    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-rbac-proxy-cm podName:53ecc34b-1bcb-4544-8c0c-a80d1c99ab04 nodeName:}" failed. No retries permitted until 2026-09-10 17:14:46.70305734 +0000 UTC m=+90.384230261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6n9rh" (UID: "53ecc34b-1bcb-4544-8c0c-a80d1c99ab04") : configmap "kube-rbac-proxy" not found
Sep 10 17:14:46.203195 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.203183    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:46.203273 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.203224    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5dtq\" (UniqueName: \"kubernetes.io/projected/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-api-access-w5dtq\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:46.203313 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.203272    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-data-volume\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:46.203355 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:46.203326    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 17:14:46.203400 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:46.203394    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls podName:53ecc34b-1bcb-4544-8c0c-a80d1c99ab04 nodeName:}" failed. No retries permitted until 2026-09-10 17:14:46.703378369 +0000 UTC m=+90.384551304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6n9rh" (UID: "53ecc34b-1bcb-4544-8c0c-a80d1c99ab04") : secret "insights-runtime-extractor-tls" not found
Sep 10 17:14:46.215170 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.215108    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5dtq\" (UniqueName: \"kubernetes.io/projected/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-api-access-w5dtq\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:46.417748 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.417656    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:46.518661 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.518617    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:46.708209 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.708080    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:46.708406 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:46.708240    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 17:14:46.708406 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.708297    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:46.708406 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:46.708316    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls podName:53ecc34b-1bcb-4544-8c0c-a80d1c99ab04 nodeName:}" failed. No retries permitted until 2026-09-10 17:14:47.708300883 +0000 UTC m=+91.389473805 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6n9rh" (UID: "53ecc34b-1bcb-4544-8c0c-a80d1c99ab04") : secret "insights-runtime-extractor-tls" not found
Sep 10 17:14:46.708406 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:46.708361    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 17:14:46.708581 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:46.708430    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-rbac-proxy-cm podName:53ecc34b-1bcb-4544-8c0c-a80d1c99ab04 nodeName:}" failed. No retries permitted until 2026-09-10 17:14:47.708414235 +0000 UTC m=+91.389587172 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6n9rh" (UID: "53ecc34b-1bcb-4544-8c0c-a80d1c99ab04") : configmap "kube-rbac-proxy" not found
Sep 10 17:14:46.718503 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.718463    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:46.819512 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.819480    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:46.955948 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:46.955909    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:47.021225 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:47.021190    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:47.716455 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:47.716396    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:47.716688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:47.716491    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:47.716688 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:47.716574    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 17:14:47.716688 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:47.716585    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 17:14:47.716688 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:47.716641    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-rbac-proxy-cm podName:53ecc34b-1bcb-4544-8c0c-a80d1c99ab04 nodeName:}" failed. No retries permitted until 2026-09-10 17:14:49.71662643 +0000 UTC m=+93.397799350 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6n9rh" (UID: "53ecc34b-1bcb-4544-8c0c-a80d1c99ab04") : configmap "kube-rbac-proxy" not found
Sep 10 17:14:47.716688 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:47.716655    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls podName:53ecc34b-1bcb-4544-8c0c-a80d1c99ab04 nodeName:}" failed. No retries permitted until 2026-09-10 17:14:49.716648945 +0000 UTC m=+93.397821866 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6n9rh" (UID: "53ecc34b-1bcb-4544-8c0c-a80d1c99ab04") : secret "insights-runtime-extractor-tls" not found
Sep 10 17:14:48.313516 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.313487    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-gvkh6"]
Sep 10 17:14:48.316390 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.316373    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-gvkh6"
Sep 10 17:14:48.320110 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.320082    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 10 17:14:48.320562 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.320522    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 10 17:14:48.320562 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.320551    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 10 17:14:48.320704 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.320584    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 10 17:14:48.320704 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.320551    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-8g2xf\""
Sep 10 17:14:48.329663 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.329633    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-gvkh6"]
Sep 10 17:14:48.355975 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.355922    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:48.421276 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.421235    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j46jg\" (UniqueName: \"kubernetes.io/projected/0a073ee6-2f01-4522-8379-7b4803cd4408-kube-api-access-j46jg\") pod \"service-ca-75d99c484-gvkh6\" (UID: \"0a073ee6-2f01-4522-8379-7b4803cd4408\") " pod="openshift-service-ca/service-ca-75d99c484-gvkh6"
Sep 10 17:14:48.421389 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.421294    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0a073ee6-2f01-4522-8379-7b4803cd4408-signing-cabundle\") pod \"service-ca-75d99c484-gvkh6\" (UID: \"0a073ee6-2f01-4522-8379-7b4803cd4408\") " pod="openshift-service-ca/service-ca-75d99c484-gvkh6"
Sep 10 17:14:48.421426 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.421406    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0a073ee6-2f01-4522-8379-7b4803cd4408-signing-key\") pod \"service-ca-75d99c484-gvkh6\" (UID: \"0a073ee6-2f01-4522-8379-7b4803cd4408\") " pod="openshift-service-ca/service-ca-75d99c484-gvkh6"
Sep 10 17:14:48.522483 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.522450    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0a073ee6-2f01-4522-8379-7b4803cd4408-signing-key\") pod \"service-ca-75d99c484-gvkh6\" (UID: \"0a073ee6-2f01-4522-8379-7b4803cd4408\") " pod="openshift-service-ca/service-ca-75d99c484-gvkh6"
Sep 10 17:14:48.522587 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.522503    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j46jg\" (UniqueName: \"kubernetes.io/projected/0a073ee6-2f01-4522-8379-7b4803cd4408-kube-api-access-j46jg\") pod \"service-ca-75d99c484-gvkh6\" (UID: \"0a073ee6-2f01-4522-8379-7b4803cd4408\") " pod="openshift-service-ca/service-ca-75d99c484-gvkh6"
Sep 10 17:14:48.522587 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.522538    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0a073ee6-2f01-4522-8379-7b4803cd4408-signing-cabundle\") pod \"service-ca-75d99c484-gvkh6\" (UID: \"0a073ee6-2f01-4522-8379-7b4803cd4408\") " pod="openshift-service-ca/service-ca-75d99c484-gvkh6"
Sep 10 17:14:48.523235 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.523205    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0a073ee6-2f01-4522-8379-7b4803cd4408-signing-cabundle\") pod \"service-ca-75d99c484-gvkh6\" (UID: \"0a073ee6-2f01-4522-8379-7b4803cd4408\") " pod="openshift-service-ca/service-ca-75d99c484-gvkh6"
Sep 10 17:14:48.525136 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.525102    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0a073ee6-2f01-4522-8379-7b4803cd4408-signing-key\") pod \"service-ca-75d99c484-gvkh6\" (UID: \"0a073ee6-2f01-4522-8379-7b4803cd4408\") " pod="openshift-service-ca/service-ca-75d99c484-gvkh6"
Sep 10 17:14:48.530907 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.530881    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j46jg\" (UniqueName: \"kubernetes.io/projected/0a073ee6-2f01-4522-8379-7b4803cd4408-kube-api-access-j46jg\") pod \"service-ca-75d99c484-gvkh6\" (UID: \"0a073ee6-2f01-4522-8379-7b4803cd4408\") " pod="openshift-service-ca/service-ca-75d99c484-gvkh6"
Sep 10 17:14:48.538832 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.538811    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fl4lm_69cd67eb-da77-4826-9bc2-a5202871c135/dns-node-resolver/0.log"
Sep 10 17:14:48.626309 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.626223    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-gvkh6"
Sep 10 17:14:48.898037 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.897961    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-gvkh6"]
Sep 10 17:14:48.901557 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:14:48.901528    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a073ee6_2f01_4522_8379_7b4803cd4408.slice/crio-837ca4483a2c61a2011f0da00da06aea38987bd3637b60f2185d65b8e2c2369c WatchSource:0}: Error finding container 837ca4483a2c61a2011f0da00da06aea38987bd3637b60f2185d65b8e2c2369c: Status 404 returned error can't find the container with id 837ca4483a2c61a2011f0da00da06aea38987bd3637b60f2185d65b8e2c2369c
Sep 10 17:14:48.926312 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:48.926271    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pvqls\" (UID: \"367ec36e-c951-4ec0-9ddc-08af4f0f6714\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pvqls"
Sep 10 17:14:48.926485 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:48.926360    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 17:14:48.926485 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:48.926420    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert podName:367ec36e-c951-4ec0-9ddc-08af4f0f6714 nodeName:}" failed. No retries permitted until 2026-09-10 17:14:56.926407234 +0000 UTC m=+100.607580159 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-pvqls" (UID: "367ec36e-c951-4ec0-9ddc-08af4f0f6714") : secret "networking-console-plugin-cert" not found
Sep 10 17:14:49.266056 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:49.266017    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-gvkh6" event={"ID":"0a073ee6-2f01-4522-8379-7b4803cd4408","Type":"ContainerStarted","Data":"837ca4483a2c61a2011f0da00da06aea38987bd3637b60f2185d65b8e2c2369c"}
Sep 10 17:14:49.541592 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:49.541510    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-plvzg_26f57ed5-a830-4098-a2ae-651e7cd00567/node-ca/0.log"
Sep 10 17:14:49.576047 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:49.575966    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:49.732923 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:49.732883    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:49.733160 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:49.732985    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:49.733160 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:49.733028    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 17:14:49.733160 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:49.733120    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-rbac-proxy-cm podName:53ecc34b-1bcb-4544-8c0c-a80d1c99ab04 nodeName:}" failed. No retries permitted until 2026-09-10 17:14:53.733098897 +0000 UTC m=+97.414271820 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6n9rh" (UID: "53ecc34b-1bcb-4544-8c0c-a80d1c99ab04") : configmap "kube-rbac-proxy" not found
Sep 10 17:14:49.733349 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:49.733169    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 17:14:49.733349 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:49.733220    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls podName:53ecc34b-1bcb-4544-8c0c-a80d1c99ab04 nodeName:}" failed. No retries permitted until 2026-09-10 17:14:53.733203987 +0000 UTC m=+97.414376908 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6n9rh" (UID: "53ecc34b-1bcb-4544-8c0c-a80d1c99ab04") : secret "insights-runtime-extractor-tls" not found
Sep 10 17:14:50.738144 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:50.738102    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vkbjg_7cf62bc7-5bc1-4f6a-b73b-a7e488ab8c6f/serve-healthcheck-canary/0.log"
Sep 10 17:14:51.273648 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:51.273611    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-gvkh6" event={"ID":"0a073ee6-2f01-4522-8379-7b4803cd4408","Type":"ContainerStarted","Data":"eaaa0aec8f87e12af6e7e49db53e871afabe3382b7f0c4ceadea79ca63f1f336"}
Sep 10 17:14:51.296661 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:51.296606    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-gvkh6" podStartSLOduration=1.5092993799999999 podStartE2EDuration="3.296589503s" totalImagesPullingTime="1.787290123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:14:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:14:51.296528256 +0000 UTC m=+94.977701198" watchObservedRunningTime="2026-09-10 17:14:51.296589503 +0000 UTC m=+94.977762445"
Sep 10 17:14:52.201799 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:52.201762    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:52.383185 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:52.383149    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:52.483514 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:52.483394    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:52.583607 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:52.583571    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:52.688210 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:52.688166    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:52.797967 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:52.797935    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:52.917667 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:52.917634    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:53.310457 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:53.310422    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:53.768750 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:53.768708    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:53.768926 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:53.768866    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:14:53.768926 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:53.768876    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 17:14:53.768996 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:53.768947    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 17:14:53.768996 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:53.768952    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls podName:53ecc34b-1bcb-4544-8c0c-a80d1c99ab04 nodeName:}" failed. No retries permitted until 2026-09-10 17:15:01.768935136 +0000 UTC m=+105.450108056 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6n9rh" (UID: "53ecc34b-1bcb-4544-8c0c-a80d1c99ab04") : secret "insights-runtime-extractor-tls" not found
Sep 10 17:14:53.768996 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:53.768988    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-rbac-proxy-cm podName:53ecc34b-1bcb-4544-8c0c-a80d1c99ab04 nodeName:}" failed. No retries permitted until 2026-09-10 17:15:01.768975724 +0000 UTC m=+105.450148645 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6n9rh" (UID: "53ecc34b-1bcb-4544-8c0c-a80d1c99ab04") : configmap "kube-rbac-proxy" not found
Sep 10 17:14:53.904177 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:53.904144    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:54.113343 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:54.113240    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:54.504941 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:54.504908    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:54.576036 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:54.575998    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:14:54.576186 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:54.576079    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls\") pod \"image-registry-8467c67c97-9dd9f\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") " pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:14:54.576186 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:54.576175    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 17:14:54.576262 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:54.576194    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 17:14:54.576262 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:54.576207    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8467c67c97-9dd9f: secret "image-registry-tls" not found
Sep 10 17:14:54.576262 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:54.576245    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls podName:2f11bf9c-45a3-4b27-8f8d-4a1d52999da2 nodeName:}" failed. No retries permitted until 2026-09-10 17:15:58.57622684 +0000 UTC m=+162.257399764 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls") pod "dns-default-pwhzb" (UID: "2f11bf9c-45a3-4b27-8f8d-4a1d52999da2") : secret "dns-default-metrics-tls" not found
Sep 10 17:14:54.576262 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:54.576259    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls podName:24d26347-fb61-4ce9-97dd-61c6c634582a nodeName:}" failed. No retries permitted until 2026-09-10 17:15:58.576252979 +0000 UTC m=+162.257425900 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls") pod "image-registry-8467c67c97-9dd9f" (UID: "24d26347-fb61-4ce9-97dd-61c6c634582a") : secret "image-registry-tls" not found
Sep 10 17:14:54.697181 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:54.697140    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:55.101023 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:55.100985    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:55.886787 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:55.886747    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:55.946795 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:55.946758    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:56.097760 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:56.097725    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:56.498764 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:56.498726    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:56.703692 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:56.703651    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:56.895035 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:56.894939    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:56.997230 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:56.997188    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pvqls\" (UID: \"367ec36e-c951-4ec0-9ddc-08af4f0f6714\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pvqls"
Sep 10 17:14:56.997459 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:56.997344    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 17:14:56.997459 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:14:56.997403    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert podName:367ec36e-c951-4ec0-9ddc-08af4f0f6714 nodeName:}" failed. No retries permitted until 2026-09-10 17:15:12.997385846 +0000 UTC m=+116.678558766 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-pvqls" (UID: "367ec36e-c951-4ec0-9ddc-08af4f0f6714") : secret "networking-console-plugin-cert" not found
Sep 10 17:14:58.006833 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:58.006782    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:58.106098 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:58.106061    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:58.299897 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:58.299805    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:58.497087 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:58.497052    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:58.597978 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:58.597872    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:58.701952 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:58.701917    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:58.899526 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:58.899428    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:14:59.219034 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:59.218948    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l4rtn"
Sep 10 17:14:59.905143 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:14:59.905091    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:00.298798 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:00.298760    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:00.500677 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:00.500634    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:00.563402 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:00.563317    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:00.705726 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:00.705688    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:01.113875 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:01.113837    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:01.723215 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:01.723178    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:01.841642 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:01.841597    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:15:01.841855 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:01.841698    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:15:01.841855 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:01.841761    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 17:15:01.841855 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:01.841832    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls podName:53ecc34b-1bcb-4544-8c0c-a80d1c99ab04 nodeName:}" failed. No retries permitted until 2026-09-10 17:15:17.841813335 +0000 UTC m=+121.522986256 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6n9rh" (UID: "53ecc34b-1bcb-4544-8c0c-a80d1c99ab04") : secret "insights-runtime-extractor-tls" not found
Sep 10 17:15:01.842304 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:01.842281    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:15:02.096358 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:02.096324    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:02.306732 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:02.306692    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:02.836407 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:02.836376    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:02.934684 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:02.934650    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:08.301668 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.301626    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:08.374107 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.374071    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:08.433080 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.433049    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66f967c844-9trnl"]
Sep 10 17:15:08.435720 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.435702    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66f967c844-9trnl"
Sep 10 17:15:08.439577 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.439547    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 17:15:08.439763 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.439691    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-2r8f8\""
Sep 10 17:15:08.439763 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.439706    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 17:15:08.439881 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.439710    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 10 17:15:08.439881 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.439691    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 17:15:08.451543 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.451497    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66f967c844-9trnl"]
Sep 10 17:15:08.585145 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.585033    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:08.596675 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.596635    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86bcf0ba-ee0c-4bf7-81ed-48313f0748fd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66f967c844-9trnl\" (UID: \"86bcf0ba-ee0c-4bf7-81ed-48313f0748fd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66f967c844-9trnl"
Sep 10 17:15:08.596867 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.596715    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j5xh\" (UniqueName: \"kubernetes.io/projected/86bcf0ba-ee0c-4bf7-81ed-48313f0748fd-kube-api-access-5j5xh\") pod \"managed-serviceaccount-addon-agent-66f967c844-9trnl\" (UID: \"86bcf0ba-ee0c-4bf7-81ed-48313f0748fd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66f967c844-9trnl"
Sep 10 17:15:08.697103 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.697061    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j5xh\" (UniqueName: \"kubernetes.io/projected/86bcf0ba-ee0c-4bf7-81ed-48313f0748fd-kube-api-access-5j5xh\") pod \"managed-serviceaccount-addon-agent-66f967c844-9trnl\" (UID: \"86bcf0ba-ee0c-4bf7-81ed-48313f0748fd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66f967c844-9trnl"
Sep 10 17:15:08.697351 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.697147    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:08.697351 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.697194    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86bcf0ba-ee0c-4bf7-81ed-48313f0748fd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66f967c844-9trnl\" (UID: \"86bcf0ba-ee0c-4bf7-81ed-48313f0748fd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66f967c844-9trnl"
Sep 10 17:15:08.699706 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.699680    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86bcf0ba-ee0c-4bf7-81ed-48313f0748fd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66f967c844-9trnl\" (UID: \"86bcf0ba-ee0c-4bf7-81ed-48313f0748fd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66f967c844-9trnl"
Sep 10 17:15:08.710366 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.710337    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j5xh\" (UniqueName: \"kubernetes.io/projected/86bcf0ba-ee0c-4bf7-81ed-48313f0748fd-kube-api-access-5j5xh\") pod \"managed-serviceaccount-addon-agent-66f967c844-9trnl\" (UID: \"86bcf0ba-ee0c-4bf7-81ed-48313f0748fd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66f967c844-9trnl"
Sep 10 17:15:08.758587 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:08.758549    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66f967c844-9trnl"
Sep 10 17:15:09.028235 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:09.028203    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66f967c844-9trnl"]
Sep 10 17:15:09.031785 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:15:09.031745    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86bcf0ba_ee0c_4bf7_81ed_48313f0748fd.slice/crio-e808d6e26cd83e892766336c6e61ad133482422d76867019fa503f136366fb75 WatchSource:0}: Error finding container e808d6e26cd83e892766336c6e61ad133482422d76867019fa503f136366fb75: Status 404 returned error can't find the container with id e808d6e26cd83e892766336c6e61ad133482422d76867019fa503f136366fb75
Sep 10 17:15:09.320539 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:09.320440    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66f967c844-9trnl" event={"ID":"86bcf0ba-ee0c-4bf7-81ed-48313f0748fd","Type":"ContainerStarted","Data":"e808d6e26cd83e892766336c6e61ad133482422d76867019fa503f136366fb75"}
Sep 10 17:15:12.331203 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:12.331164    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66f967c844-9trnl" event={"ID":"86bcf0ba-ee0c-4bf7-81ed-48313f0748fd","Type":"ContainerStarted","Data":"6ba5114f431bdfd724e01f4c4c434cc25c590271a9a1e07da0708b13a4b51aff"}
Sep 10 17:15:12.347316 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:12.347262    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66f967c844-9trnl" podStartSLOduration=1.7512276039999999 podStartE2EDuration="4.347241916s" totalImagesPullingTime="2.596014312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:15:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:15:12.346931232 +0000 UTC m=+116.028104177" watchObservedRunningTime="2026-09-10 17:15:12.347241916 +0000 UTC m=+116.028414858"
Sep 10 17:15:13.031997 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:13.031957    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pvqls\" (UID: \"367ec36e-c951-4ec0-9ddc-08af4f0f6714\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pvqls"
Sep 10 17:15:13.034661 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:13.034624    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/367ec36e-c951-4ec0-9ddc-08af4f0f6714-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pvqls\" (UID: \"367ec36e-c951-4ec0-9ddc-08af4f0f6714\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pvqls"
Sep 10 17:15:13.278392 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:13.278333    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-pvqls"
Sep 10 17:15:13.550459 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:13.550370    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-pvqls"]
Sep 10 17:15:13.554235 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:15:13.554190    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod367ec36e_c951_4ec0_9ddc_08af4f0f6714.slice/crio-485eb815a21c85f289299152b58a3af03a7e7f9506b161c64a257bc9afe4caf3 WatchSource:0}: Error finding container 485eb815a21c85f289299152b58a3af03a7e7f9506b161c64a257bc9afe4caf3: Status 404 returned error can't find the container with id 485eb815a21c85f289299152b58a3af03a7e7f9506b161c64a257bc9afe4caf3
Sep 10 17:15:14.338600 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:14.338557    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-pvqls" event={"ID":"367ec36e-c951-4ec0-9ddc-08af4f0f6714","Type":"ContainerStarted","Data":"485eb815a21c85f289299152b58a3af03a7e7f9506b161c64a257bc9afe4caf3"}
Sep 10 17:15:15.343253 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:15.343212    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-pvqls" event={"ID":"367ec36e-c951-4ec0-9ddc-08af4f0f6714","Type":"ContainerStarted","Data":"1978555a2f6cc10aca1c10ebce0932501fd367e8c497cfc04b5c27a5833cabd4"}
Sep 10 17:15:15.566872 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:15.566835    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:15.778707 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:15.778676    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:15.915984 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:15.915954    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:15.954849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:15.954794    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-pvqls" podStartSLOduration=33.957424397 podStartE2EDuration="34.954776221s" totalImagesPullingTime="997.351824ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:14:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:15:15.363645692 +0000 UTC m=+119.044818635" watchObservedRunningTime="2026-09-10 17:15:15.954776221 +0000 UTC m=+119.635949204"
Sep 10 17:15:15.955578 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:15.955557    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-76wwx"]
Sep 10 17:15:15.958955 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:15.958934    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-76wwx"
Sep 10 17:15:15.963866 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:15.963836    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-2jmcm\""
Sep 10 17:15:15.964148 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:15.964114    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 10 17:15:15.970818 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:15.970796    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-76wwx"]
Sep 10 17:15:16.012054 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:16.012013    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:16.055430 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:16.055326    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e1226059-45ac-43b9-8a04-8c47c900178e-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-76wwx\" (UID: \"e1226059-45ac-43b9-8a04-8c47c900178e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-76wwx"
Sep 10 17:15:16.116290 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:16.116248    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:16.156328 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:16.156289    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e1226059-45ac-43b9-8a04-8c47c900178e-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-76wwx\" (UID: \"e1226059-45ac-43b9-8a04-8c47c900178e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-76wwx"
Sep 10 17:15:16.158842 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:16.158812    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e1226059-45ac-43b9-8a04-8c47c900178e-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-76wwx\" (UID: \"e1226059-45ac-43b9-8a04-8c47c900178e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-76wwx"
Sep 10 17:15:16.270430 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:16.270382    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-76wwx"
Sep 10 17:15:16.548331 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:16.548299    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-76wwx"]
Sep 10 17:15:16.551540 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:15:16.551512    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1226059_45ac_43b9_8a04_8c47c900178e.slice/crio-0e5f77afb361889a9b78578f570e4f94cd9def3e68a71057e0254330f982e728 WatchSource:0}: Error finding container 0e5f77afb361889a9b78578f570e4f94cd9def3e68a71057e0254330f982e728: Status 404 returned error can't find the container with id 0e5f77afb361889a9b78578f570e4f94cd9def3e68a71057e0254330f982e728
Sep 10 17:15:17.351119 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:17.351082    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-76wwx" event={"ID":"e1226059-45ac-43b9-8a04-8c47c900178e","Type":"ContainerStarted","Data":"0e5f77afb361889a9b78578f570e4f94cd9def3e68a71057e0254330f982e728"}
Sep 10 17:15:17.870318 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:17.870205    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:15:17.872682 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:17.872648    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53ecc34b-1bcb-4544-8c0c-a80d1c99ab04-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6n9rh\" (UID: \"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04\") " pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:15:18.127048 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:18.126941    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2hvq7\""
Sep 10 17:15:18.134463 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:18.134430    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6n9rh"
Sep 10 17:15:18.355800 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:18.355759    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-76wwx" event={"ID":"e1226059-45ac-43b9-8a04-8c47c900178e","Type":"ContainerStarted","Data":"1f25c477913ca9c8be69e938cd19c565ee5ee7fa5fead894452752255c307768"}
Sep 10 17:15:18.356018 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:18.355973    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-76wwx"
Sep 10 17:15:18.361068 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:18.361042    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-76wwx"
Sep 10 17:15:18.378894 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:18.378766    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-76wwx" podStartSLOduration=2.339070447 podStartE2EDuration="3.378751491s" totalImagesPullingTime="1.039681044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:15:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:15:18.378102467 +0000 UTC m=+122.059275420" watchObservedRunningTime="2026-09-10 17:15:18.378751491 +0000 UTC m=+122.059924434"
Sep 10 17:15:18.424340 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:18.424302    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6n9rh"]
Sep 10 17:15:18.427696 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:15:18.427666    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53ecc34b_1bcb_4544_8c0c_a80d1c99ab04.slice/crio-eba6543165ac68bff32298a600d4d922bd920d06b31d79defc638532607ffea7 WatchSource:0}: Error finding container eba6543165ac68bff32298a600d4d922bd920d06b31d79defc638532607ffea7: Status 404 returned error can't find the container with id eba6543165ac68bff32298a600d4d922bd920d06b31d79defc638532607ffea7
Sep 10 17:15:19.039485 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.039438    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:19.142647 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.142609    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-pcljz"]
Sep 10 17:15:19.145801 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.145774    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz"
Sep 10 17:15:19.151694 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.151446    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 17:15:19.151694 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.151515    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 10 17:15:19.151694 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.151531    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-vfqxf\""
Sep 10 17:15:19.151694 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.151572    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 17:15:19.151694 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.151531    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 17:15:19.152120 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.151883    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 10 17:15:19.162574 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.162543    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-pcljz"]
Sep 10 17:15:19.178752 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.178719    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:19.282836 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.282795    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hclhk\" (UniqueName: \"kubernetes.io/projected/fb2e3d7c-ddea-4060-ba1b-4450b7522775-kube-api-access-hclhk\") pod \"prometheus-operator-6f4946db56-pcljz\" (UID: \"fb2e3d7c-ddea-4060-ba1b-4450b7522775\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz"
Sep 10 17:15:19.283012 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.282863    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb2e3d7c-ddea-4060-ba1b-4450b7522775-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-pcljz\" (UID: \"fb2e3d7c-ddea-4060-ba1b-4450b7522775\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz"
Sep 10 17:15:19.283012 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.282897    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb2e3d7c-ddea-4060-ba1b-4450b7522775-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-pcljz\" (UID: \"fb2e3d7c-ddea-4060-ba1b-4450b7522775\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz"
Sep 10 17:15:19.283012 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.282952    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb2e3d7c-ddea-4060-ba1b-4450b7522775-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pcljz\" (UID: \"fb2e3d7c-ddea-4060-ba1b-4450b7522775\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz"
Sep 10 17:15:19.360752 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.360645    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6n9rh" event={"ID":"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04","Type":"ContainerStarted","Data":"c73077db9a25a23f8d872dd51a78adb937e7f7d6ddc55b0ce045d0628d523679"}
Sep 10 17:15:19.360752 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.360716    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6n9rh" event={"ID":"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04","Type":"ContainerStarted","Data":"d2933665e6b24448662bc3663a405a6e0b63c4e27e81dd23c4d2c15c5dbadfb4"}
Sep 10 17:15:19.360752 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.360751    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6n9rh" event={"ID":"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04","Type":"ContainerStarted","Data":"eba6543165ac68bff32298a600d4d922bd920d06b31d79defc638532607ffea7"}
Sep 10 17:15:19.384340 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.384303    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb2e3d7c-ddea-4060-ba1b-4450b7522775-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-pcljz\" (UID: \"fb2e3d7c-ddea-4060-ba1b-4450b7522775\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz"
Sep 10 17:15:19.384487 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.384349    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb2e3d7c-ddea-4060-ba1b-4450b7522775-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-pcljz\" (UID: \"fb2e3d7c-ddea-4060-ba1b-4450b7522775\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz"
Sep 10 17:15:19.384487 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.384368    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb2e3d7c-ddea-4060-ba1b-4450b7522775-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pcljz\" (UID: \"fb2e3d7c-ddea-4060-ba1b-4450b7522775\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz"
Sep 10 17:15:19.384487 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:19.384479    2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 10 17:15:19.384640 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.384494    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hclhk\" (UniqueName: \"kubernetes.io/projected/fb2e3d7c-ddea-4060-ba1b-4450b7522775-kube-api-access-hclhk\") pod \"prometheus-operator-6f4946db56-pcljz\" (UID: \"fb2e3d7c-ddea-4060-ba1b-4450b7522775\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz"
Sep 10 17:15:19.384640 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:19.384532    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb2e3d7c-ddea-4060-ba1b-4450b7522775-prometheus-operator-tls podName:fb2e3d7c-ddea-4060-ba1b-4450b7522775 nodeName:}" failed. No retries permitted until 2026-09-10 17:15:19.884519043 +0000 UTC m=+123.565691964 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/fb2e3d7c-ddea-4060-ba1b-4450b7522775-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-pcljz" (UID: "fb2e3d7c-ddea-4060-ba1b-4450b7522775") : secret "prometheus-operator-tls" not found
Sep 10 17:15:19.385148 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.385110    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb2e3d7c-ddea-4060-ba1b-4450b7522775-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-pcljz\" (UID: \"fb2e3d7c-ddea-4060-ba1b-4450b7522775\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz"
Sep 10 17:15:19.386852 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.386831    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb2e3d7c-ddea-4060-ba1b-4450b7522775-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-pcljz\" (UID: \"fb2e3d7c-ddea-4060-ba1b-4450b7522775\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz"
Sep 10 17:15:19.394160 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.394109    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hclhk\" (UniqueName: \"kubernetes.io/projected/fb2e3d7c-ddea-4060-ba1b-4450b7522775-kube-api-access-hclhk\") pod \"prometheus-operator-6f4946db56-pcljz\" (UID: \"fb2e3d7c-ddea-4060-ba1b-4450b7522775\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz"
Sep 10 17:15:19.581375 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.581338    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:19.888943 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.888897    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb2e3d7c-ddea-4060-ba1b-4450b7522775-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pcljz\" (UID: \"fb2e3d7c-ddea-4060-ba1b-4450b7522775\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz"
Sep 10 17:15:19.891740 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:19.891711    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb2e3d7c-ddea-4060-ba1b-4450b7522775-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pcljz\" (UID: \"fb2e3d7c-ddea-4060-ba1b-4450b7522775\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz"
Sep 10 17:15:20.054837 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:20.054797    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz"
Sep 10 17:15:20.348514 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:20.348468    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-pcljz"]
Sep 10 17:15:20.555825 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:15:20.555774    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb2e3d7c_ddea_4060_ba1b_4450b7522775.slice/crio-1e8b907bb33c80102e800ea60fb719eed5b1e1f0be50679db7fcc16834a7eaf7 WatchSource:0}: Error finding container 1e8b907bb33c80102e800ea60fb719eed5b1e1f0be50679db7fcc16834a7eaf7: Status 404 returned error can't find the container with id 1e8b907bb33c80102e800ea60fb719eed5b1e1f0be50679db7fcc16834a7eaf7
Sep 10 17:15:21.370968 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:21.370926    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6n9rh" event={"ID":"53ecc34b-1bcb-4544-8c0c-a80d1c99ab04","Type":"ContainerStarted","Data":"8999acb66a028b1c70a7b8f8537481975a235f14bdc40ddc0f888147ddbf0843"}
Sep 10 17:15:21.372370 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:21.372341    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz" event={"ID":"fb2e3d7c-ddea-4060-ba1b-4450b7522775","Type":"ContainerStarted","Data":"1e8b907bb33c80102e800ea60fb719eed5b1e1f0be50679db7fcc16834a7eaf7"}
Sep 10 17:15:21.398535 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:21.398478    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6n9rh" podStartSLOduration=34.344531885 podStartE2EDuration="36.398454019s" totalImagesPullingTime="2.053922134s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:14:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:15:21.397295164 +0000 UTC m=+125.078468162" watchObservedRunningTime="2026-09-10 17:15:21.398454019 +0000 UTC m=+125.079626964"
Sep 10 17:15:21.989963 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:21.989859    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:22.253761 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:22.253668    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:22.377496 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:22.377458    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz" event={"ID":"fb2e3d7c-ddea-4060-ba1b-4450b7522775","Type":"ContainerStarted","Data":"dba9feb9212cf83e75c67194a7efa41f7925c5bbe18afbb078708140309b0631"}
Sep 10 17:15:22.377892 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:22.377542    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz" event={"ID":"fb2e3d7c-ddea-4060-ba1b-4450b7522775","Type":"ContainerStarted","Data":"44ccfdd402d4ef24fb20762e3658c79dde43debc6cbac5dcce6f31e441374f56"}
Sep 10 17:15:22.416209 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:22.416154    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-pcljz" podStartSLOduration=2.336241342 podStartE2EDuration="3.416115958s" totalImagesPullingTime="1.079874616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:15:22.415522088 +0000 UTC m=+126.096695030" watchObservedRunningTime="2026-09-10 17:15:22.416115958 +0000 UTC m=+126.097288901"
Sep 10 17:15:22.474569 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:22.474531    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:24.273564 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.273528    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:24.554084 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.553997    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:24.625366 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.625331    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5"]
Sep 10 17:15:24.630741 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.630715    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5"
Sep 10 17:15:24.638332 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.638309    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 10 17:15:24.638462 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.638309    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-lhf5p\""
Sep 10 17:15:24.638518 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.638473    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 10 17:15:24.664597 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.664558    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5"]
Sep 10 17:15:24.680077 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.680043    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"]
Sep 10 17:15:24.684449 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.684426    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.687780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.687748    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 10 17:15:24.687910 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.687787    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 10 17:15:24.688086 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.688071    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 10 17:15:24.688721 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.688703    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-5jnhs\""
Sep 10 17:15:24.708455 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.708421    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"]
Sep 10 17:15:24.711929 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.711896    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:24.722936 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.722898    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9w7qm"]
Sep 10 17:15:24.727196 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.727176    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.730165 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.730139    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/baf56765-367e-4fb5-82ed-838f05e2818c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-5txp5\" (UID: \"baf56765-367e-4fb5-82ed-838f05e2818c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5"
Sep 10 17:15:24.730331 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.730175    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/baf56765-367e-4fb5-82ed-838f05e2818c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-5txp5\" (UID: \"baf56765-367e-4fb5-82ed-838f05e2818c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5"
Sep 10 17:15:24.730331 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.730218    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/baf56765-367e-4fb5-82ed-838f05e2818c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-5txp5\" (UID: \"baf56765-367e-4fb5-82ed-838f05e2818c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5"
Sep 10 17:15:24.730331 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.730241    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bde2e973-b9b2-4b62-ad5d-1eb467ef1210-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-l4b56\" (UID: \"bde2e973-b9b2-4b62-ad5d-1eb467ef1210\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.730331 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.730295    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bde2e973-b9b2-4b62-ad5d-1eb467ef1210-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-l4b56\" (UID: \"bde2e973-b9b2-4b62-ad5d-1eb467ef1210\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.730331 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.730318    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgz9s\" (UniqueName: \"kubernetes.io/projected/bde2e973-b9b2-4b62-ad5d-1eb467ef1210-kube-api-access-xgz9s\") pod \"kube-state-metrics-74cb57df7-l4b56\" (UID: \"bde2e973-b9b2-4b62-ad5d-1eb467ef1210\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.730584 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.730389    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/bde2e973-b9b2-4b62-ad5d-1eb467ef1210-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-l4b56\" (UID: \"bde2e973-b9b2-4b62-ad5d-1eb467ef1210\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.730584 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.730456    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w957h\" (UniqueName: \"kubernetes.io/projected/baf56765-367e-4fb5-82ed-838f05e2818c-kube-api-access-w957h\") pod \"openshift-state-metrics-657fd6c54-5txp5\" (UID: \"baf56765-367e-4fb5-82ed-838f05e2818c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5"
Sep 10 17:15:24.730584 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.730499    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bde2e973-b9b2-4b62-ad5d-1eb467ef1210-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-l4b56\" (UID: \"bde2e973-b9b2-4b62-ad5d-1eb467ef1210\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.730584 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.730520    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/bde2e973-b9b2-4b62-ad5d-1eb467ef1210-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-l4b56\" (UID: \"bde2e973-b9b2-4b62-ad5d-1eb467ef1210\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.730895 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.730878    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 17:15:24.730953 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.730904    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 17:15:24.730953 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.730924    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 17:15:24.731048 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.730955    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gkjjw\""
Sep 10 17:15:24.830912 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.830816    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/baf56765-367e-4fb5-82ed-838f05e2818c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-5txp5\" (UID: \"baf56765-367e-4fb5-82ed-838f05e2818c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5"
Sep 10 17:15:24.830912 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.830850    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bde2e973-b9b2-4b62-ad5d-1eb467ef1210-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-l4b56\" (UID: \"bde2e973-b9b2-4b62-ad5d-1eb467ef1210\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.830912 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.830883    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bde2e973-b9b2-4b62-ad5d-1eb467ef1210-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-l4b56\" (UID: \"bde2e973-b9b2-4b62-ad5d-1eb467ef1210\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.830912 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.830905    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgz9s\" (UniqueName: \"kubernetes.io/projected/bde2e973-b9b2-4b62-ad5d-1eb467ef1210-kube-api-access-xgz9s\") pod \"kube-state-metrics-74cb57df7-l4b56\" (UID: \"bde2e973-b9b2-4b62-ad5d-1eb467ef1210\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.831269 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.830925    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/30d419bb-93c9-465c-b86c-31291a08dd78-metrics-client-ca\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.831269 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.831070    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/30d419bb-93c9-465c-b86c-31291a08dd78-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.831269 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.831144    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/30d419bb-93c9-465c-b86c-31291a08dd78-node-exporter-wtmp\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.831269 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.831184    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/bde2e973-b9b2-4b62-ad5d-1eb467ef1210-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-l4b56\" (UID: \"bde2e973-b9b2-4b62-ad5d-1eb467ef1210\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.831269 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.831220    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/30d419bb-93c9-465c-b86c-31291a08dd78-node-exporter-tls\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.831575 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.831283    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w957h\" (UniqueName: \"kubernetes.io/projected/baf56765-367e-4fb5-82ed-838f05e2818c-kube-api-access-w957h\") pod \"openshift-state-metrics-657fd6c54-5txp5\" (UID: \"baf56765-367e-4fb5-82ed-838f05e2818c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5"
Sep 10 17:15:24.831575 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.831324    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bde2e973-b9b2-4b62-ad5d-1eb467ef1210-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-l4b56\" (UID: \"bde2e973-b9b2-4b62-ad5d-1eb467ef1210\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.831679 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.831587    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/30d419bb-93c9-465c-b86c-31291a08dd78-node-exporter-accelerators-collector-config\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.831679 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.831624    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/bde2e973-b9b2-4b62-ad5d-1eb467ef1210-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-l4b56\" (UID: \"bde2e973-b9b2-4b62-ad5d-1eb467ef1210\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.831679 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.831654    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/30d419bb-93c9-465c-b86c-31291a08dd78-root\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.831817 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.831680    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/30d419bb-93c9-465c-b86c-31291a08dd78-node-exporter-textfile\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.831817 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.831741    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsnpl\" (UniqueName: \"kubernetes.io/projected/30d419bb-93c9-465c-b86c-31291a08dd78-kube-api-access-wsnpl\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.831817 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.831781    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/baf56765-367e-4fb5-82ed-838f05e2818c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-5txp5\" (UID: \"baf56765-367e-4fb5-82ed-838f05e2818c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5"
Sep 10 17:15:24.831961 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.831824    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/baf56765-367e-4fb5-82ed-838f05e2818c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-5txp5\" (UID: \"baf56765-367e-4fb5-82ed-838f05e2818c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5"
Sep 10 17:15:24.831961 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.831850    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30d419bb-93c9-465c-b86c-31291a08dd78-sys\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.831961 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.831943    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/bde2e973-b9b2-4b62-ad5d-1eb467ef1210-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-l4b56\" (UID: \"bde2e973-b9b2-4b62-ad5d-1eb467ef1210\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.832112 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.831992    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/bde2e973-b9b2-4b62-ad5d-1eb467ef1210-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-l4b56\" (UID: \"bde2e973-b9b2-4b62-ad5d-1eb467ef1210\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.832319 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.832294    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bde2e973-b9b2-4b62-ad5d-1eb467ef1210-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-l4b56\" (UID: \"bde2e973-b9b2-4b62-ad5d-1eb467ef1210\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.832555 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.832536    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/baf56765-367e-4fb5-82ed-838f05e2818c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-5txp5\" (UID: \"baf56765-367e-4fb5-82ed-838f05e2818c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5"
Sep 10 17:15:24.834296 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.834263    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/baf56765-367e-4fb5-82ed-838f05e2818c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-5txp5\" (UID: \"baf56765-367e-4fb5-82ed-838f05e2818c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5"
Sep 10 17:15:24.834408 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.834293    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bde2e973-b9b2-4b62-ad5d-1eb467ef1210-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-l4b56\" (UID: \"bde2e973-b9b2-4b62-ad5d-1eb467ef1210\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.834408 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.834382    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bde2e973-b9b2-4b62-ad5d-1eb467ef1210-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-l4b56\" (UID: \"bde2e973-b9b2-4b62-ad5d-1eb467ef1210\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.838849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.838824    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/baf56765-367e-4fb5-82ed-838f05e2818c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-5txp5\" (UID: \"baf56765-367e-4fb5-82ed-838f05e2818c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5"
Sep 10 17:15:24.844189 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.844156    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w957h\" (UniqueName: \"kubernetes.io/projected/baf56765-367e-4fb5-82ed-838f05e2818c-kube-api-access-w957h\") pod \"openshift-state-metrics-657fd6c54-5txp5\" (UID: \"baf56765-367e-4fb5-82ed-838f05e2818c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5"
Sep 10 17:15:24.844296 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.844197    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgz9s\" (UniqueName: \"kubernetes.io/projected/bde2e973-b9b2-4b62-ad5d-1eb467ef1210-kube-api-access-xgz9s\") pod \"kube-state-metrics-74cb57df7-l4b56\" (UID: \"bde2e973-b9b2-4b62-ad5d-1eb467ef1210\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:24.852973 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.852946    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:24.932921 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.932876    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/30d419bb-93c9-465c-b86c-31291a08dd78-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.932921 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.932921    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/30d419bb-93c9-465c-b86c-31291a08dd78-node-exporter-wtmp\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.933179 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.932946    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/30d419bb-93c9-465c-b86c-31291a08dd78-node-exporter-tls\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.933179 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.932987    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/30d419bb-93c9-465c-b86c-31291a08dd78-node-exporter-accelerators-collector-config\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.933179 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.933010    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/30d419bb-93c9-465c-b86c-31291a08dd78-root\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.933179 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.933027    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/30d419bb-93c9-465c-b86c-31291a08dd78-node-exporter-textfile\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.933179 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.933044    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsnpl\" (UniqueName: \"kubernetes.io/projected/30d419bb-93c9-465c-b86c-31291a08dd78-kube-api-access-wsnpl\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.933179 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.933074    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30d419bb-93c9-465c-b86c-31291a08dd78-sys\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.933179 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.933082    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/30d419bb-93c9-465c-b86c-31291a08dd78-node-exporter-wtmp\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.933179 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.933097    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/30d419bb-93c9-465c-b86c-31291a08dd78-root\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.933179 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.933161    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30d419bb-93c9-465c-b86c-31291a08dd78-sys\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.933179 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.933177    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/30d419bb-93c9-465c-b86c-31291a08dd78-metrics-client-ca\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.933675 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.933536    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/30d419bb-93c9-465c-b86c-31291a08dd78-node-exporter-textfile\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.933775 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.933754    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/30d419bb-93c9-465c-b86c-31291a08dd78-node-exporter-accelerators-collector-config\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.933846 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.933822    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/30d419bb-93c9-465c-b86c-31291a08dd78-metrics-client-ca\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.935423 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.935393    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/30d419bb-93c9-465c-b86c-31291a08dd78-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.935608 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.935587    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/30d419bb-93c9-465c-b86c-31291a08dd78-node-exporter-tls\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.940563 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.940535    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5"
Sep 10 17:15:24.956787 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.956758    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsnpl\" (UniqueName: \"kubernetes.io/projected/30d419bb-93c9-465c-b86c-31291a08dd78-kube-api-access-wsnpl\") pod \"node-exporter-9w7qm\" (UID: \"30d419bb-93c9-465c-b86c-31291a08dd78\") " pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:24.975762 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.975714    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:24.994487 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:24.994322    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"
Sep 10 17:15:25.036861 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:25.036814    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9w7qm"
Sep 10 17:15:25.051227 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:15:25.050607    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30d419bb_93c9_465c_b86c_31291a08dd78.slice/crio-1da47691f49d03ce995e46f369f738d2d3fc7fab015fa641ff2f4d5c54078c65 WatchSource:0}: Error finding container 1da47691f49d03ce995e46f369f738d2d3fc7fab015fa641ff2f4d5c54078c65: Status 404 returned error can't find the container with id 1da47691f49d03ce995e46f369f738d2d3fc7fab015fa641ff2f4d5c54078c65
Sep 10 17:15:25.126757 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:25.126662    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:25.221927 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:25.221889    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5"]
Sep 10 17:15:25.225709 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:15:25.225676    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaf56765_367e_4fb5_82ed_838f05e2818c.slice/crio-9b8bdd8a4e012fcef9c357c95faab76cecb11f31e7119c2cf3d8be7593fd0aea WatchSource:0}: Error finding container 9b8bdd8a4e012fcef9c357c95faab76cecb11f31e7119c2cf3d8be7593fd0aea: Status 404 returned error can't find the container with id 9b8bdd8a4e012fcef9c357c95faab76cecb11f31e7119c2cf3d8be7593fd0aea
Sep 10 17:15:25.293054 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:25.293031    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-l4b56"]
Sep 10 17:15:25.297209 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:15:25.297175    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbde2e973_b9b2_4b62_ad5d_1eb467ef1210.slice/crio-274149b05424082505a731a96bc7cf2b0d2a97f2b69d132df877ce1b377c3339 WatchSource:0}: Error finding container 274149b05424082505a731a96bc7cf2b0d2a97f2b69d132df877ce1b377c3339: Status 404 returned error can't find the container with id 274149b05424082505a731a96bc7cf2b0d2a97f2b69d132df877ce1b377c3339
Sep 10 17:15:25.389624 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:25.389588    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56" event={"ID":"bde2e973-b9b2-4b62-ad5d-1eb467ef1210","Type":"ContainerStarted","Data":"274149b05424082505a731a96bc7cf2b0d2a97f2b69d132df877ce1b377c3339"}
Sep 10 17:15:25.391840 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:25.391796    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5" event={"ID":"baf56765-367e-4fb5-82ed-838f05e2818c","Type":"ContainerStarted","Data":"ceaac3ac87ae3bd0c93c112f78d05f3f7aadd030a1a7cbede2e73881b63dfd53"}
Sep 10 17:15:25.391996 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:25.391872    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5" event={"ID":"baf56765-367e-4fb5-82ed-838f05e2818c","Type":"ContainerStarted","Data":"850cf43b999008241308f8e98014eaa39e6ce2d63d9f94ea643087ce4690bcce"}
Sep 10 17:15:25.391996 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:25.391893    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5" event={"ID":"baf56765-367e-4fb5-82ed-838f05e2818c","Type":"ContainerStarted","Data":"9b8bdd8a4e012fcef9c357c95faab76cecb11f31e7119c2cf3d8be7593fd0aea"}
Sep 10 17:15:25.393179 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:25.393155    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9w7qm" event={"ID":"30d419bb-93c9-465c-b86c-31291a08dd78","Type":"ContainerStarted","Data":"1da47691f49d03ce995e46f369f738d2d3fc7fab015fa641ff2f4d5c54078c65"}
Sep 10 17:15:25.630101 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:25.630063    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:25.830184 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:25.830120    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:26.048785 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.048748    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:26.085336 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.085255    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 17:15:26.089475 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.089442    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.101808 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:26.101705    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-main-tls-assets-0\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" type="*v1.Secret"
Sep 10 17:15:26.101808 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:26.101725    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-main-web-config\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" type="*v1.Secret"
Sep 10 17:15:26.101808 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:26.101735    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"alertmanager-trusted-ca-bundle\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" type="*v1.ConfigMap"
Sep 10 17:15:26.101808 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:26.101735    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-main-generated\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" type="*v1.Secret"
Sep 10 17:15:26.101808 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:26.101734    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"alertmanager-main-0\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.101808 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:26.101781    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-kube-rbac-proxy\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" type="*v1.Secret"
Sep 10 17:15:26.101808 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:26.101781    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-main-dockercfg-2whcg\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2whcg\"" type="*v1.Secret"
Sep 10 17:15:26.101808 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:26.101799    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-kube-rbac-proxy-metric\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" type="*v1.Secret"
Sep 10 17:15:26.101808 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:26.101806    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-kube-rbac-proxy-web\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" type="*v1.Secret"
Sep 10 17:15:26.105896 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:26.105859    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-main-cluster-tls-config\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" type="*v1.Secret"
Sep 10 17:15:26.106841 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:26.106804    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-main-tls\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" type="*v1.Secret"
Sep 10 17:15:26.144928 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.144883    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.145097 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.144947    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/78250d22-60e4-45ec-b893-2edd9dd8a390-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.145097 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.145014    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-config-volume\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.145097 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.145089    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.145403 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.145118    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-web-config\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.145403 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.145215    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.145403 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.145259    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/78250d22-60e4-45ec-b893-2edd9dd8a390-config-out\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.145403 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.145288    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/78250d22-60e4-45ec-b893-2edd9dd8a390-tls-assets\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.145403 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.145313    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78250d22-60e4-45ec-b893-2edd9dd8a390-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.145403 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.145353    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78250d22-60e4-45ec-b893-2edd9dd8a390-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.145403 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.145382    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.145651 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.145410    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf9rr\" (UniqueName: \"kubernetes.io/projected/78250d22-60e4-45ec-b893-2edd9dd8a390-kube-api-access-xf9rr\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.145651 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.145490    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.161296 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.161223    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 17:15:26.246403 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.246359    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/78250d22-60e4-45ec-b893-2edd9dd8a390-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.246581 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.246428    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-config-volume\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.246641 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.246608    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.246727 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.246661    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-web-config\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.246727 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.246690    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.246826 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.246791    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/78250d22-60e4-45ec-b893-2edd9dd8a390-config-out\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.246873 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.246825    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/78250d22-60e4-45ec-b893-2edd9dd8a390-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.246873 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.246832    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/78250d22-60e4-45ec-b893-2edd9dd8a390-tls-assets\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.246979 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.246883    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78250d22-60e4-45ec-b893-2edd9dd8a390-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.246979 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.246911    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78250d22-60e4-45ec-b893-2edd9dd8a390-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.246979 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.246936    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.246979 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.246953    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf9rr\" (UniqueName: \"kubernetes.io/projected/78250d22-60e4-45ec-b893-2edd9dd8a390-kube-api-access-xf9rr\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.247203 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.247002    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.247203 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.247086    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.247623 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.247601    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78250d22-60e4-45ec-b893-2edd9dd8a390-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.249654 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.249626    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/78250d22-60e4-45ec-b893-2edd9dd8a390-config-out\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.260901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.260868    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf9rr\" (UniqueName: \"kubernetes.io/projected/78250d22-60e4-45ec-b893-2edd9dd8a390-kube-api-access-xf9rr\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:26.351042 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.350952    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:26.550588 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.550545    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs\") pod \"network-metrics-daemon-rzqqc\" (UID: \"7a5b4316-2391-4e12-9a80-12ace78ee49d\") " pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:15:26.553565 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.553524    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5b4316-2391-4e12-9a80-12ace78ee49d-metrics-certs\") pod \"network-metrics-daemon-rzqqc\" (UID: \"7a5b4316-2391-4e12-9a80-12ace78ee49d\") " pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:15:26.862603 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.862508    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n4rqr\""
Sep 10 17:15:26.872157 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.872113    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzqqc"
Sep 10 17:15:26.950383 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.950344    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 17:15:26.959134 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:26.959092    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-web-config\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:27.166683 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:15:27.166642    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a5b4316_2391_4e12_9a80_12ace78ee49d.slice/crio-a33ca9eee474033dcf0471dcb291d1a1ce594b42b6d51e90da20f4ecdcfa882d WatchSource:0}: Error finding container a33ca9eee474033dcf0471dcb291d1a1ce594b42b6d51e90da20f4ecdcfa882d: Status 404 returned error can't find the container with id a33ca9eee474033dcf0471dcb291d1a1ce594b42b6d51e90da20f4ecdcfa882d
Sep 10 17:15:27.174110 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.174083    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 17:15:27.178827 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.178797    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 17:15:27.179463 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.179441    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-config-volume\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:27.187323 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.187240    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rzqqc"]
Sep 10 17:15:27.187993 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.187971    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78250d22-60e4-45ec-b893-2edd9dd8a390-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:27.206232 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.206200    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2whcg\""
Sep 10 17:15:27.246925 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.246887    2566 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-cluster-tls-config: failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:27.247171 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.246946    2566 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:27.247171 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.246962    2566 projected.go:266] Couldn't get secret openshift-monitoring/alertmanager-main-tls-assets-0: failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:27.247171 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.246979    2566 projected.go:196] Error preparing data for projected volume tls-assets for pod openshift-monitoring/alertmanager-main-0: failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:27.247171 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.246993    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-cluster-tls-config podName:78250d22-60e4-45ec-b893-2edd9dd8a390 nodeName:}" failed. No retries permitted until 2026-09-10 17:15:27.74697067 +0000 UTC m=+131.428143611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-tls-config" (UniqueName: "kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-cluster-tls-config") pod "alertmanager-main-0" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390") : failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:27.247171 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.247013    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-main-tls podName:78250d22-60e4-45ec-b893-2edd9dd8a390 nodeName:}" failed. No retries permitted until 2026-09-10 17:15:27.74700455 +0000 UTC m=+131.428177470 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390") : failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:27.247171 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.247035    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78250d22-60e4-45ec-b893-2edd9dd8a390-tls-assets podName:78250d22-60e4-45ec-b893-2edd9dd8a390 nodeName:}" failed. No retries permitted until 2026-09-10 17:15:27.747029248 +0000 UTC m=+131.428202169 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-assets" (UniqueName: "kubernetes.io/projected/78250d22-60e4-45ec-b893-2edd9dd8a390-tls-assets") pod "alertmanager-main-0" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390") : failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:27.247171 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.247157    2566 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-kube-rbac-proxy: failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:27.247537 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.247197    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy podName:78250d22-60e4-45ec-b893-2edd9dd8a390 nodeName:}" failed. No retries permitted until 2026-09-10 17:15:27.747188526 +0000 UTC m=+131.428361446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-kube-rbac-proxy" (UniqueName: "kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy") pod "alertmanager-main-0" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390") : failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:27.247537 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.247232    2566 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-kube-rbac-proxy-metric: failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:27.247537 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.247267    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy-metric podName:78250d22-60e4-45ec-b893-2edd9dd8a390 nodeName:}" failed. No retries permitted until 2026-09-10 17:15:27.747257467 +0000 UTC m=+131.428430404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-kube-rbac-proxy-metric" (UniqueName: "kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy-metric") pod "alertmanager-main-0" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390") : failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:27.247537 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.247291    2566 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-kube-rbac-proxy-web: failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:27.247537 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.247342    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy-web podName:78250d22-60e4-45ec-b893-2edd9dd8a390 nodeName:}" failed. No retries permitted until 2026-09-10 17:15:27.747328343 +0000 UTC m=+131.428501267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-kube-rbac-proxy-web" (UniqueName: "kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy-web") pod "alertmanager-main-0" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390") : failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:27.251526 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.251492    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 17:15:27.273013 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.272983    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 17:15:27.362699 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.362660    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 17:15:27.403633 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.403581    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56" event={"ID":"bde2e973-b9b2-4b62-ad5d-1eb467ef1210","Type":"ContainerStarted","Data":"befa1b2b7d64532fbf27363a7ead0be5868ceaf496e0cf11109c93e7dcf4e4d8"}
Sep 10 17:15:27.403848 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.403662    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56" event={"ID":"bde2e973-b9b2-4b62-ad5d-1eb467ef1210","Type":"ContainerStarted","Data":"80123659b35e46ee08c67877794711b89356036ce27384408018e65fb1ebe45d"}
Sep 10 17:15:27.403848 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.403694    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56" event={"ID":"bde2e973-b9b2-4b62-ad5d-1eb467ef1210","Type":"ContainerStarted","Data":"8d29c5346c81f55fd21dd100289742efc468ad8c0d5721ec842391e81ef6b918"}
Sep 10 17:15:27.406268 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.406222    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5" event={"ID":"baf56765-367e-4fb5-82ed-838f05e2818c","Type":"ContainerStarted","Data":"4cc09c13548f29bc407ecd5f6b92d50a901150770aaaa00f8b95cd067096ef08"}
Sep 10 17:15:27.407528 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.407504    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rzqqc" event={"ID":"7a5b4316-2391-4e12-9a80-12ace78ee49d","Type":"ContainerStarted","Data":"a33ca9eee474033dcf0471dcb291d1a1ce594b42b6d51e90da20f4ecdcfa882d"}
Sep 10 17:15:27.408066 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.408047    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 17:15:27.409382 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.409348    2566 generic.go:356] "Generic (PLEG): container finished" podID="30d419bb-93c9-465c-b86c-31291a08dd78" containerID="a2d20b24d336915412fb52a2c0b3a315a4ebe52dcb350de4df6fbe868e67ffdd" exitCode=0
Sep 10 17:15:27.409480 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.409413    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9w7qm" event={"ID":"30d419bb-93c9-465c-b86c-31291a08dd78","Type":"ContainerDied","Data":"a2d20b24d336915412fb52a2c0b3a315a4ebe52dcb350de4df6fbe868e67ffdd"}
Sep 10 17:15:27.419889 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.419868    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 17:15:27.437176 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.437105    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-l4b56" podStartSLOduration=2.054001014 podStartE2EDuration="3.437089139s" totalImagesPullingTime="1.383088125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:15:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:15:27.436202005 +0000 UTC m=+131.117374950" watchObservedRunningTime="2026-09-10 17:15:27.437089139 +0000 UTC m=+131.118262081"
Sep 10 17:15:27.548186 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.548096    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5txp5" podStartSLOduration=2.245438081 podStartE2EDuration="3.548076667s" totalImagesPullingTime="1.302638586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:15:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:15:27.546893408 +0000 UTC m=+131.228066350" watchObservedRunningTime="2026-09-10 17:15:27.548076667 +0000 UTC m=+131.229249610"
Sep 10 17:15:27.691652 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.691614    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 17:15:27.765466 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.765430    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:27.765646 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.765505    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:27.765646 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.765555    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:27.765646 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.765588    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/78250d22-60e4-45ec-b893-2edd9dd8a390-tls-assets\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:27.765646 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.765621    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:27.765854 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.765648    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:27.768849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.768816    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:27.768849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.768839    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:27.769062 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.768823    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/78250d22-60e4-45ec-b893-2edd9dd8a390-tls-assets\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:27.769062 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.768871    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:27.769206 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.769100    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:27.769300 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.769269    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:27.901392 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.901356    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:15:27.909749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.909714    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"]
Sep 10 17:15:27.914897 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.914874    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:27.925830 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.925215    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"thanos-querier-tls\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" type="*v1.Secret"
Sep 10 17:15:27.925830 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.925338    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"thanos-querier-grpc-tls-ck3cbeovdsfr9\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-ck3cbeovdsfr9\"" type="*v1.Secret"
Sep 10 17:15:27.925830 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.925386    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"thanos-querier-kube-rbac-proxy-metrics\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" type="*v1.Secret"
Sep 10 17:15:27.925830 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.925480    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"thanos-querier-7bc4ff5bff-dcvn2\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" podUID="b743a223-6782-45ce-84c5-cefdfca6ed76" pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:27.925830 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.925544    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"thanos-querier-dockercfg-ck8fc\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ck8fc\"" type="*v1.Secret"
Sep 10 17:15:27.925830 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.925683    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"thanos-querier-kube-rbac-proxy-web\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" type="*v1.Secret"
Sep 10 17:15:27.925830 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.925802    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"thanos-querier-kube-rbac-proxy\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" type="*v1.Secret"
Sep 10 17:15:27.926534 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:27.926503    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"thanos-querier-kube-rbac-proxy-rules\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" type="*v1.Secret"
Sep 10 17:15:27.961957 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.961918    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"]
Sep 10 17:15:27.967804 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.967765    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b743a223-6782-45ce-84c5-cefdfca6ed76-metrics-client-ca\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:27.967930 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.967843    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs2cs\" (UniqueName: \"kubernetes.io/projected/b743a223-6782-45ce-84c5-cefdfca6ed76-kube-api-access-gs2cs\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:27.968003 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.967942    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:27.968003 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.967972    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:27.968109 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.968004    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:27.968109 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.968065    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:27.968407 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.968168    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-tls\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:27.968407 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:27.968209    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-grpc-tls\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:28.069215 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.069116    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-grpc-tls\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:28.069215 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.069182    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b743a223-6782-45ce-84c5-cefdfca6ed76-metrics-client-ca\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:28.069215 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.069213    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs2cs\" (UniqueName: \"kubernetes.io/projected/b743a223-6782-45ce-84c5-cefdfca6ed76-kube-api-access-gs2cs\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:28.069478 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.069350    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:28.069478 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.069395    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:28.069478 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.069429    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:28.069633 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.069506    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:28.069633 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.069566    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-tls\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:28.069940 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.069922    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b743a223-6782-45ce-84c5-cefdfca6ed76-metrics-client-ca\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:28.086671 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.086642    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs2cs\" (UniqueName: \"kubernetes.io/projected/b743a223-6782-45ce-84c5-cefdfca6ed76-kube-api-access-gs2cs\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:28.226779 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.226740    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 17:15:28.307992 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:15:28.307960    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78250d22_60e4_45ec_b893_2edd9dd8a390.slice/crio-e26cb21fc5b8dc3c22de2114088785686489c29a58f0cb938d14064347562a7f WatchSource:0}: Error finding container e26cb21fc5b8dc3c22de2114088785686489c29a58f0cb938d14064347562a7f: Status 404 returned error can't find the container with id e26cb21fc5b8dc3c22de2114088785686489c29a58f0cb938d14064347562a7f
Sep 10 17:15:28.323433 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.323344    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:28.417446 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.417154    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9w7qm" event={"ID":"30d419bb-93c9-465c-b86c-31291a08dd78","Type":"ContainerStarted","Data":"f679621fb585c5e4c337ab9a5381d2ebf32cc0ea3db3ed420048d72016ee08d4"}
Sep 10 17:15:28.417446 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.417251    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9w7qm" event={"ID":"30d419bb-93c9-465c-b86c-31291a08dd78","Type":"ContainerStarted","Data":"574e98ad4c715aac1a4d57759d1c593bed22bfc5c885703d3fb28178f2d9ff36"}
Sep 10 17:15:28.424773 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.424089    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"78250d22-60e4-45ec-b893-2edd9dd8a390","Type":"ContainerStarted","Data":"e26cb21fc5b8dc3c22de2114088785686489c29a58f0cb938d14064347562a7f"}
Sep 10 17:15:28.448240 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.448188    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9w7qm" podStartSLOduration=2.7436901049999998 podStartE2EDuration="4.448171351s" totalImagesPullingTime="1.623885065s" totalInitContainerRuntime="80.596181ms" isStatefulPod=true podCreationTimestamp="2026-09-10 17:15:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:15:28.447922384 +0000 UTC m=+132.129095339" watchObservedRunningTime="2026-09-10 17:15:28.448171351 +0000 UTC m=+132.129344306"
Sep 10 17:15:28.454711 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.454674    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:28.670418 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.670329    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:28.830452 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.830397    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:28.980186 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.980085    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 10 17:15:28.992991 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:28.992951    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:29.035239 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.035208    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 10 17:15:29.042322 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.042289    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:29.069508 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:29.069467    2566 secret.go:189] Couldn't get secret openshift-monitoring/thanos-querier-grpc-tls-ck3cbeovdsfr9: failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:29.069702 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:29.069569    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-grpc-tls podName:b743a223-6782-45ce-84c5-cefdfca6ed76 nodeName:}" failed. No retries permitted until 2026-09-10 17:15:29.569543812 +0000 UTC m=+133.250716732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-grpc-tls" (UniqueName: "kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-grpc-tls") pod "thanos-querier-7bc4ff5bff-dcvn2" (UID: "b743a223-6782-45ce-84c5-cefdfca6ed76") : failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:29.069702 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:29.069602    2566 secret.go:189] Couldn't get secret openshift-monitoring/thanos-querier-kube-rbac-proxy-web: failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:29.069702 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:29.069694    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-kube-rbac-proxy-web podName:b743a223-6782-45ce-84c5-cefdfca6ed76 nodeName:}" failed. No retries permitted until 2026-09-10 17:15:29.569668425 +0000 UTC m=+133.250841347 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-thanos-querier-kube-rbac-proxy-web" (UniqueName: "kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-kube-rbac-proxy-web") pod "thanos-querier-7bc4ff5bff-dcvn2" (UID: "b743a223-6782-45ce-84c5-cefdfca6ed76") : failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:29.069702 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:29.069698    2566 secret.go:189] Couldn't get secret openshift-monitoring/thanos-querier-kube-rbac-proxy: failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:29.069927 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:29.069787    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-kube-rbac-proxy podName:b743a223-6782-45ce-84c5-cefdfca6ed76 nodeName:}" failed. No retries permitted until 2026-09-10 17:15:29.569769792 +0000 UTC m=+133.250942725 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-thanos-querier-kube-rbac-proxy" (UniqueName: "kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-kube-rbac-proxy") pod "thanos-querier-7bc4ff5bff-dcvn2" (UID: "b743a223-6782-45ce-84c5-cefdfca6ed76") : failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:29.069927 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:29.069812    2566 secret.go:189] Couldn't get secret openshift-monitoring/thanos-querier-tls: failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:29.069927 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:29.069871    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-tls podName:b743a223-6782-45ce-84c5-cefdfca6ed76 nodeName:}" failed. No retries permitted until 2026-09-10 17:15:29.569853131 +0000 UTC m=+133.251026058 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-thanos-querier-tls" (UniqueName: "kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-tls") pod "thanos-querier-7bc4ff5bff-dcvn2" (UID: "b743a223-6782-45ce-84c5-cefdfca6ed76") : failed to sync secret cache: timed out waiting for the condition
Sep 10 17:15:29.127659 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.127629    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ck8fc\""
Sep 10 17:15:29.231904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.231835    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 10 17:15:29.361361 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.361330    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-ck3cbeovdsfr9\""
Sep 10 17:15:29.363616 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.363584    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 10 17:15:29.379303 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.379269    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:29.381867 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.381844    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 10 17:15:29.431378 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.431344    2566 generic.go:356] "Generic (PLEG): container finished" podID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerID="3dd6db0af04abf78c709262d6be0563b41423ea141e58eefeb63050214519e80" exitCode=0
Sep 10 17:15:29.431836 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.431432    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"78250d22-60e4-45ec-b893-2edd9dd8a390","Type":"ContainerDied","Data":"3dd6db0af04abf78c709262d6be0563b41423ea141e58eefeb63050214519e80"}
Sep 10 17:15:29.433711 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.433685    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rzqqc" event={"ID":"7a5b4316-2391-4e12-9a80-12ace78ee49d","Type":"ContainerStarted","Data":"d968975adbf8779c5d2f7699d7855627bc35bfe4808f9c6c759a3ff46660aa03"}
Sep 10 17:15:29.433822 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.433742    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rzqqc" event={"ID":"7a5b4316-2391-4e12-9a80-12ace78ee49d","Type":"ContainerStarted","Data":"7699a2576f168da10bd3f08392983d157865196abf91f8a2d40063ed5453b64b"}
Sep 10 17:15:29.447367 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.447338    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-mkm8q"]
Sep 10 17:15:29.450982 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.450965    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mkm8q"
Sep 10 17:15:29.460335 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.460310    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-f7vpl\""
Sep 10 17:15:29.460751 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.460732    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 10 17:15:29.470786 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.470758    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-mkm8q"]
Sep 10 17:15:29.485672 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.485578    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c28pf\" (UniqueName: \"kubernetes.io/projected/9fd7049a-7081-4356-9111-58beb92aa7b3-kube-api-access-c28pf\") pod \"monitoring-plugin-85bbbbbf4-mkm8q\" (UID: \"9fd7049a-7081-4356-9111-58beb92aa7b3\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mkm8q"
Sep 10 17:15:29.485843 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.485786    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fd7049a-7081-4356-9111-58beb92aa7b3-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-mkm8q\" (UID: \"9fd7049a-7081-4356-9111-58beb92aa7b3\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mkm8q"
Sep 10 17:15:29.583303 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.583237    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rzqqc" podStartSLOduration=132.396794225 podStartE2EDuration="2m13.583212992s" totalImagesPullingTime="1.186418767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:13:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:15:29.582677894 +0000 UTC m=+133.263850836" watchObservedRunningTime="2026-09-10 17:15:29.583212992 +0000 UTC m=+133.264385936"
Sep 10 17:15:29.587014 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.586982    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:29.587211 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.587079    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-tls\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:29.587211 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.587119    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-grpc-tls\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:29.587211 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.587206    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fd7049a-7081-4356-9111-58beb92aa7b3-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-mkm8q\" (UID: \"9fd7049a-7081-4356-9111-58beb92aa7b3\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mkm8q"
Sep 10 17:15:29.587361 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.587291    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:29.587361 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.587338    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c28pf\" (UniqueName: \"kubernetes.io/projected/9fd7049a-7081-4356-9111-58beb92aa7b3-kube-api-access-c28pf\") pod \"monitoring-plugin-85bbbbbf4-mkm8q\" (UID: \"9fd7049a-7081-4356-9111-58beb92aa7b3\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mkm8q"
Sep 10 17:15:29.587451 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:29.587416    2566 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 10 17:15:29.587531 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:29.587494    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd7049a-7081-4356-9111-58beb92aa7b3-monitoring-plugin-cert podName:9fd7049a-7081-4356-9111-58beb92aa7b3 nodeName:}" failed. No retries permitted until 2026-09-10 17:15:30.087474277 +0000 UTC m=+133.768647209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/9fd7049a-7081-4356-9111-58beb92aa7b3-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-mkm8q" (UID: "9fd7049a-7081-4356-9111-58beb92aa7b3") : secret "monitoring-plugin-cert" not found
Sep 10 17:15:29.589920 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.589891    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:29.590014 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.589930    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-grpc-tls\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:29.590049 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.590018    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:29.590093 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.590022    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b743a223-6782-45ce-84c5-cefdfca6ed76-secret-thanos-querier-tls\") pod \"thanos-querier-7bc4ff5bff-dcvn2\" (UID: \"b743a223-6782-45ce-84c5-cefdfca6ed76\") " pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:29.620237 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.620202    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c28pf\" (UniqueName: \"kubernetes.io/projected/9fd7049a-7081-4356-9111-58beb92aa7b3-kube-api-access-c28pf\") pod \"monitoring-plugin-85bbbbbf4-mkm8q\" (UID: \"9fd7049a-7081-4356-9111-58beb92aa7b3\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mkm8q"
Sep 10 17:15:29.736978 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.736871    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:29.835099 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.831837    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:29.959894 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:29.959836    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:30.048196 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:30.048086    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"]
Sep 10 17:15:30.053203 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:15:30.053147    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb743a223_6782_45ce_84c5_cefdfca6ed76.slice/crio-4c8697d94d9f12e03db5286ef5cfe067299f766bc03658fe85fc36e126d48a8d WatchSource:0}: Error finding container 4c8697d94d9f12e03db5286ef5cfe067299f766bc03658fe85fc36e126d48a8d: Status 404 returned error can't find the container with id 4c8697d94d9f12e03db5286ef5cfe067299f766bc03658fe85fc36e126d48a8d
Sep 10 17:15:30.093050 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:30.093003    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fd7049a-7081-4356-9111-58beb92aa7b3-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-mkm8q\" (UID: \"9fd7049a-7081-4356-9111-58beb92aa7b3\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mkm8q"
Sep 10 17:15:30.096367 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:30.096330    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fd7049a-7081-4356-9111-58beb92aa7b3-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-mkm8q\" (UID: \"9fd7049a-7081-4356-9111-58beb92aa7b3\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mkm8q"
Sep 10 17:15:30.361154 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:30.361039    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mkm8q"
Sep 10 17:15:30.439007 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:30.438961    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2" event={"ID":"b743a223-6782-45ce-84c5-cefdfca6ed76","Type":"ContainerStarted","Data":"4c8697d94d9f12e03db5286ef5cfe067299f766bc03658fe85fc36e126d48a8d"}
Sep 10 17:15:30.783454 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:30.783413    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8467c67c97-9dd9f"]
Sep 10 17:15:30.783765 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:30.783745    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-8467c67c97-9dd9f" podUID="24d26347-fb61-4ce9-97dd-61c6c634582a"
Sep 10 17:15:30.833459 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:15:30.833418    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd7049a_7081_4356_9111_58beb92aa7b3.slice/crio-dda963cdfaa46f57f0d880c38cf43cc574af068b18bd5d56269aa5c3f1f99c0c WatchSource:0}: Error finding container dda963cdfaa46f57f0d880c38cf43cc574af068b18bd5d56269aa5c3f1f99c0c: Status 404 returned error can't find the container with id dda963cdfaa46f57f0d880c38cf43cc574af068b18bd5d56269aa5c3f1f99c0c
Sep 10 17:15:30.833918 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:30.833892    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-mkm8q"]
Sep 10 17:15:30.905409 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:30.905338    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:30.999469 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:30.999431    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:31.139499 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.139407    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:31.258918 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.258879    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:31.447321 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.447169    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mkm8q" event={"ID":"9fd7049a-7081-4356-9111-58beb92aa7b3","Type":"ContainerStarted","Data":"dda963cdfaa46f57f0d880c38cf43cc574af068b18bd5d56269aa5c3f1f99c0c"}
Sep 10 17:15:31.452286 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.452109    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"78250d22-60e4-45ec-b893-2edd9dd8a390","Type":"ContainerStarted","Data":"4c92ffcbe8757554760f9a8960004819c68c6477767754befbefaae5531b01e8"}
Sep 10 17:15:31.452286 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.452163    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:15:31.452286 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.452216    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"78250d22-60e4-45ec-b893-2edd9dd8a390","Type":"ContainerStarted","Data":"86888fbea2e780249d15627ae6dc0e72673fb251f18cf2254b521029f4246a68"}
Sep 10 17:15:31.452286 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.452259    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"78250d22-60e4-45ec-b893-2edd9dd8a390","Type":"ContainerStarted","Data":"0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3"}
Sep 10 17:15:31.452637 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.452331    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"78250d22-60e4-45ec-b893-2edd9dd8a390","Type":"ContainerStarted","Data":"f0c5bbe03139619cdf3887d81fafbf20d78ed5d2c414c3f63d694a419c4db00e"}
Sep 10 17:15:31.452637 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.452369    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"78250d22-60e4-45ec-b893-2edd9dd8a390","Type":"ContainerStarted","Data":"fb082b4bb45e1082f08bda14f6a96f24f738a13e7717c20571dc88d8da99771a"}
Sep 10 17:15:31.463601 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.463566    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:15:31.508413 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.508367    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24d26347-fb61-4ce9-97dd-61c6c634582a-ca-trust-extracted\") pod \"24d26347-fb61-4ce9-97dd-61c6c634582a\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") "
Sep 10 17:15:31.508614 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.508432    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24d26347-fb61-4ce9-97dd-61c6c634582a-installation-pull-secrets\") pod \"24d26347-fb61-4ce9-97dd-61c6c634582a\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") "
Sep 10 17:15:31.508614 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.508484    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24d26347-fb61-4ce9-97dd-61c6c634582a-image-registry-private-configuration\") pod \"24d26347-fb61-4ce9-97dd-61c6c634582a\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") "
Sep 10 17:15:31.508614 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.508524    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-bound-sa-token\") pod \"24d26347-fb61-4ce9-97dd-61c6c634582a\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") "
Sep 10 17:15:31.508614 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.508553    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-certificates\") pod \"24d26347-fb61-4ce9-97dd-61c6c634582a\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") "
Sep 10 17:15:31.508614 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.508587    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cx9l\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-kube-api-access-4cx9l\") pod \"24d26347-fb61-4ce9-97dd-61c6c634582a\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") "
Sep 10 17:15:31.508912 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.508637    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24d26347-fb61-4ce9-97dd-61c6c634582a-trusted-ca\") pod \"24d26347-fb61-4ce9-97dd-61c6c634582a\" (UID: \"24d26347-fb61-4ce9-97dd-61c6c634582a\") "
Sep 10 17:15:31.508912 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.508727    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24d26347-fb61-4ce9-97dd-61c6c634582a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "24d26347-fb61-4ce9-97dd-61c6c634582a" (UID: "24d26347-fb61-4ce9-97dd-61c6c634582a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:15:31.509041 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.509002    2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24d26347-fb61-4ce9-97dd-61c6c634582a-ca-trust-extracted\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:15:31.509518 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.509469    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24d26347-fb61-4ce9-97dd-61c6c634582a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "24d26347-fb61-4ce9-97dd-61c6c634582a" (UID: "24d26347-fb61-4ce9-97dd-61c6c634582a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:15:31.509660 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.509608    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "24d26347-fb61-4ce9-97dd-61c6c634582a" (UID: "24d26347-fb61-4ce9-97dd-61c6c634582a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:15:31.511710 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.511679    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24d26347-fb61-4ce9-97dd-61c6c634582a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "24d26347-fb61-4ce9-97dd-61c6c634582a" (UID: "24d26347-fb61-4ce9-97dd-61c6c634582a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:15:31.512031 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.511986    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "24d26347-fb61-4ce9-97dd-61c6c634582a" (UID: "24d26347-fb61-4ce9-97dd-61c6c634582a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:15:31.512170 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.512053    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-kube-api-access-4cx9l" (OuterVolumeSpecName: "kube-api-access-4cx9l") pod "24d26347-fb61-4ce9-97dd-61c6c634582a" (UID: "24d26347-fb61-4ce9-97dd-61c6c634582a"). InnerVolumeSpecName "kube-api-access-4cx9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:15:31.512268 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.512245    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24d26347-fb61-4ce9-97dd-61c6c634582a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "24d26347-fb61-4ce9-97dd-61c6c634582a" (UID: "24d26347-fb61-4ce9-97dd-61c6c634582a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:15:31.610170 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.610105    2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-bound-sa-token\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:15:31.610170 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.610160    2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-certificates\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:15:31.610170 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.610176    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cx9l\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-kube-api-access-4cx9l\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:15:31.610453 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.610189    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24d26347-fb61-4ce9-97dd-61c6c634582a-trusted-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:15:31.610453 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.610204    2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24d26347-fb61-4ce9-97dd-61c6c634582a-installation-pull-secrets\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:15:31.610453 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.610220    2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24d26347-fb61-4ce9-97dd-61c6c634582a-image-registry-private-configuration\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:15:31.734310 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:31.734163    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:32.279597 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:32.279560    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:32.393624 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:32.393568    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:32.456245 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:32.456215    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8467c67c97-9dd9f"
Sep 10 17:15:32.504015 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:32.503966    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8467c67c97-9dd9f"]
Sep 10 17:15:32.512875 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:32.512840    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8467c67c97-9dd9f"]
Sep 10 17:15:32.620360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:32.620275    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24d26347-fb61-4ce9-97dd-61c6c634582a-registry-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:15:32.836178 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:32.836142    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24d26347-fb61-4ce9-97dd-61c6c634582a" path="/var/lib/kubelet/pods/24d26347-fb61-4ce9-97dd-61c6c634582a/volumes"
Sep 10 17:15:33.029487 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:33.029453    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:33.133433 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:33.133341    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:33.462143 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:33.462016    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mkm8q" event={"ID":"9fd7049a-7081-4356-9111-58beb92aa7b3","Type":"ContainerStarted","Data":"35877bf5f8b90fdc2dbe352b3c1d67616cc7d85aeb60fc7ff07af6651c086121"}
Sep 10 17:15:33.462143 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:33.462088    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mkm8q"
Sep 10 17:15:33.465740 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:33.465702    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2" event={"ID":"b743a223-6782-45ce-84c5-cefdfca6ed76","Type":"ContainerStarted","Data":"09316e7f928b0d646159bfeb8c75a9865921b1a3dcb135993a09e54b31bd5087"}
Sep 10 17:15:33.465909 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:33.465791    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2" event={"ID":"b743a223-6782-45ce-84c5-cefdfca6ed76","Type":"ContainerStarted","Data":"ed491c47a8915befc23cbde45e15989f26f0be802c9dcd7e3faddc524e238044"}
Sep 10 17:15:33.465909 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:33.465833    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2" event={"ID":"b743a223-6782-45ce-84c5-cefdfca6ed76","Type":"ContainerStarted","Data":"fa0d305719f1b90e34f024066581940847f6fc61495e79c223adba623ae81ecd"}
Sep 10 17:15:33.465909 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:33.465869    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2" event={"ID":"b743a223-6782-45ce-84c5-cefdfca6ed76","Type":"ContainerStarted","Data":"f5f80bbdec1a77f9116aae12cad1f18c67ff0688117e67ae009deb44cfc9e611"}
Sep 10 17:15:33.466045 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:33.465913    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:33.466045 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:33.465957    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2" event={"ID":"b743a223-6782-45ce-84c5-cefdfca6ed76","Type":"ContainerStarted","Data":"77ea500fd0bdaa9b90e31532e225666cca99efe8176fa252395b946235dbfe2c"}
Sep 10 17:15:33.466045 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:33.465996    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2" event={"ID":"b743a223-6782-45ce-84c5-cefdfca6ed76","Type":"ContainerStarted","Data":"abe53773aa1bd5f257e91bc2a5a97dd1e854245e09c8d76bcf1702308e7b52d0"}
Sep 10 17:15:33.468209 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:33.468186    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mkm8q"
Sep 10 17:15:33.469369 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:33.469343    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"78250d22-60e4-45ec-b893-2edd9dd8a390","Type":"ContainerStarted","Data":"837d25db185e428ca6a7f45b7232e39a85b33e4dd8b4cd04a3dddb26bfbc7306"}
Sep 10 17:15:33.478428 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:33.478367    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mkm8q" podStartSLOduration=2.434311337 podStartE2EDuration="4.478349398s" totalImagesPullingTime="2.044038061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:15:33.477887739 +0000 UTC m=+137.159060682" watchObservedRunningTime="2026-09-10 17:15:33.478349398 +0000 UTC m=+137.159522342"
Sep 10 17:15:33.507709 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:33.507656    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.496967997 podStartE2EDuration="7.507637122s" totalImagesPullingTime="3.993681504s" totalInitContainerRuntime="16.987621ms" isStatefulPod=false podCreationTimestamp="2026-09-10 17:15:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:15:33.505848175 +0000 UTC m=+137.187021164" watchObservedRunningTime="2026-09-10 17:15:33.507637122 +0000 UTC m=+137.188810066"
Sep 10 17:15:33.532957 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:33.532901    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2" podStartSLOduration=3.704202407 podStartE2EDuration="6.532883088s" totalImagesPullingTime="2.828680681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:15:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:15:33.530999072 +0000 UTC m=+137.212172016" watchObservedRunningTime="2026-09-10 17:15:33.532883088 +0000 UTC m=+137.214056027"
Sep 10 17:15:36.037203 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:36.037162    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:36.135735 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:36.135698    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:38.280933 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:38.280897    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-5mpq7"]
Sep 10 17:15:38.284288 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:38.284259    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-5mpq7"
Sep 10 17:15:38.287045 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:38.287020    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 17:15:38.287314 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:38.287296    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-8h6mc\""
Sep 10 17:15:38.287414 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:38.287337    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 17:15:38.296481 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:38.296454    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-5mpq7"]
Sep 10 17:15:38.297499 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:38.297466    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:38.372835 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:38.372794    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l52v9\" (UniqueName: \"kubernetes.io/projected/e6d42206-f79c-43c6-bdc4-991cd18fd0cd-kube-api-access-l52v9\") pod \"downloads-85d98c998f-5mpq7\" (UID: \"e6d42206-f79c-43c6-bdc4-991cd18fd0cd\") " pod="openshift-console/downloads-85d98c998f-5mpq7"
Sep 10 17:15:38.396091 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:38.396051    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:38.473787 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:38.473734    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l52v9\" (UniqueName: \"kubernetes.io/projected/e6d42206-f79c-43c6-bdc4-991cd18fd0cd-kube-api-access-l52v9\") pod \"downloads-85d98c998f-5mpq7\" (UID: \"e6d42206-f79c-43c6-bdc4-991cd18fd0cd\") " pod="openshift-console/downloads-85d98c998f-5mpq7"
Sep 10 17:15:38.482787 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:38.482750    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l52v9\" (UniqueName: \"kubernetes.io/projected/e6d42206-f79c-43c6-bdc4-991cd18fd0cd-kube-api-access-l52v9\") pod \"downloads-85d98c998f-5mpq7\" (UID: \"e6d42206-f79c-43c6-bdc4-991cd18fd0cd\") " pod="openshift-console/downloads-85d98c998f-5mpq7"
Sep 10 17:15:38.496624 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:38.496580    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:38.593974 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:38.593877    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:38.594604 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:38.594572    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-5mpq7"
Sep 10 17:15:38.877251 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:38.877058    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-5mpq7"]
Sep 10 17:15:38.879782 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:15:38.879755    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6d42206_f79c_43c6_bdc4_991cd18fd0cd.slice/crio-b244e490a7d1db32eb60316af3b8fc309c80060d6a5bcf3c7ef0298cd57950f0 WatchSource:0}: Error finding container b244e490a7d1db32eb60316af3b8fc309c80060d6a5bcf3c7ef0298cd57950f0: Status 404 returned error can't find the container with id b244e490a7d1db32eb60316af3b8fc309c80060d6a5bcf3c7ef0298cd57950f0
Sep 10 17:15:38.973192 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:38.973152    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:39.041284 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:39.041245    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:39.361259 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:39.361226    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:39.438537 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:39.438485    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:39.480522 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:39.480493    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7bc4ff5bff-dcvn2"
Sep 10 17:15:39.494734 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:39.494695    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-5mpq7" event={"ID":"e6d42206-f79c-43c6-bdc4-991cd18fd0cd","Type":"ContainerStarted","Data":"b244e490a7d1db32eb60316af3b8fc309c80060d6a5bcf3c7ef0298cd57950f0"}
Sep 10 17:15:39.560048 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:39.560013    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:39.660498 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:39.660399    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:40.042969 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:40.042934    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:40.200091 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:40.200030    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:40.402155 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:40.402044    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:41.585827 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:41.585781    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:41.687988 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:41.687951    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:44.148815 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:44.148767    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:44.224516 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:44.224472    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:48.244365 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.244332    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57d44b69d6-v8tm8"]
Sep 10 17:15:48.252852 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.252822    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.256521 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.256455    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 17:15:48.256521 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.256467    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 17:15:48.256753 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.256526    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-xbw8p\""
Sep 10 17:15:48.256753 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.256525    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 17:15:48.256753 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.256602    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 17:15:48.256753 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.256467    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 17:15:48.264706 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.264680    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57d44b69d6-v8tm8"]
Sep 10 17:15:48.370455 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.370404    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-console-serving-cert\") pod \"console-57d44b69d6-v8tm8\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") " pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.370647 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.370477    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-service-ca\") pod \"console-57d44b69d6-v8tm8\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") " pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.370647 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.370540    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-oauth-serving-cert\") pod \"console-57d44b69d6-v8tm8\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") " pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.370768 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.370640    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-console-oauth-config\") pod \"console-57d44b69d6-v8tm8\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") " pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.370768 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.370700    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4kfz\" (UniqueName: \"kubernetes.io/projected/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-kube-api-access-k4kfz\") pod \"console-57d44b69d6-v8tm8\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") " pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.370858 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.370801    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-console-config\") pod \"console-57d44b69d6-v8tm8\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") " pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.472037 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.471991    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-console-serving-cert\") pod \"console-57d44b69d6-v8tm8\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") " pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.472228 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.472051    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-service-ca\") pod \"console-57d44b69d6-v8tm8\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") " pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.472228 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.472079    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-oauth-serving-cert\") pod \"console-57d44b69d6-v8tm8\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") " pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.472228 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.472112    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-console-oauth-config\") pod \"console-57d44b69d6-v8tm8\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") " pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.472403 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.472272    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4kfz\" (UniqueName: \"kubernetes.io/projected/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-kube-api-access-k4kfz\") pod \"console-57d44b69d6-v8tm8\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") " pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.472403 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.472364    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-console-config\") pod \"console-57d44b69d6-v8tm8\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") " pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.472928 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.472893    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-service-ca\") pod \"console-57d44b69d6-v8tm8\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") " pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.473059 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.473037    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-console-config\") pod \"console-57d44b69d6-v8tm8\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") " pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.473148 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.473094    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-oauth-serving-cert\") pod \"console-57d44b69d6-v8tm8\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") " pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.474847 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.474825    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-console-oauth-config\") pod \"console-57d44b69d6-v8tm8\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") " pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.474968 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.474935    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-console-serving-cert\") pod \"console-57d44b69d6-v8tm8\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") " pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.482509 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.482478    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4kfz\" (UniqueName: \"kubernetes.io/projected/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-kube-api-access-k4kfz\") pod \"console-57d44b69d6-v8tm8\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") " pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:48.566708 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:48.566609    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:15:52.993140 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:52.993079    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:53.092804 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:53.092732    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:53.308000 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:53.307898    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:53.375703 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:53.375655    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:53.722368 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:15:53.722273    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-pwhzb" podUID="2f11bf9c-45a3-4b27-8f8d-4a1d52999da2"
Sep 10 17:15:54.571806 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:54.571771    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:15:55.173275 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.173234    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78fc566c97-xwzzz"]
Sep 10 17:15:55.179986 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.179952    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.192631 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.192600    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 17:15:55.212091 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.212061    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78fc566c97-xwzzz"]
Sep 10 17:15:55.342158 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.342103    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-console-config\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.342291 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.342206    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-service-ca\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.342291 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.342237    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-console-serving-cert\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.342291 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.342262    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-console-oauth-config\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.342291 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.342279    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-trusted-ca-bundle\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.342448 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.342387    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq8wl\" (UniqueName: \"kubernetes.io/projected/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-kube-api-access-tq8wl\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.342492 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.342444    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-oauth-serving-cert\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.443430 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.443342    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-service-ca\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.443430 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.443390    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-console-serving-cert\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.443430 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.443411    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-console-oauth-config\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.443656 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.443433    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-trusted-ca-bundle\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.443656 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.443584    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq8wl\" (UniqueName: \"kubernetes.io/projected/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-kube-api-access-tq8wl\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.443756 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.443667    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-oauth-serving-cert\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.443756 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.443739    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-console-config\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.444265 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.444236    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-service-ca\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.444410 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.444359    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-trusted-ca-bundle\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.444481 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.444437    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-console-config\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.444481 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.444446    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-oauth-serving-cert\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.446010 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.445982    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-console-serving-cert\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.446120 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.446097    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-console-oauth-config\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.453723 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.453673    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq8wl\" (UniqueName: \"kubernetes.io/projected/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-kube-api-access-tq8wl\") pod \"console-78fc566c97-xwzzz\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") " pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.479675 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.479632    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57d44b69d6-v8tm8"]
Sep 10 17:15:55.483474 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:15:55.483432    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ae8d927_c657_4f1a_83ab_a4e596ec21bc.slice/crio-6f4891dfd773f1217eeeb79eb39ae017a8b07541b46828ac221643fc2c8bc72c WatchSource:0}: Error finding container 6f4891dfd773f1217eeeb79eb39ae017a8b07541b46828ac221643fc2c8bc72c: Status 404 returned error can't find the container with id 6f4891dfd773f1217eeeb79eb39ae017a8b07541b46828ac221643fc2c8bc72c
Sep 10 17:15:55.492852 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.492815    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:15:55.579066 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.578962    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d44b69d6-v8tm8" event={"ID":"3ae8d927-c657-4f1a-83ab-a4e596ec21bc","Type":"ContainerStarted","Data":"6f4891dfd773f1217eeeb79eb39ae017a8b07541b46828ac221643fc2c8bc72c"}
Sep 10 17:15:55.581943 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.581879    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-5mpq7" event={"ID":"e6d42206-f79c-43c6-bdc4-991cd18fd0cd","Type":"ContainerStarted","Data":"dedbe8c65500956e6e061cccbe3e64d20102787619ff2394381c6cf53a282d8c"}
Sep 10 17:15:55.582358 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.582315    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-5mpq7"
Sep 10 17:15:55.592689 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.592652    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-5mpq7"
Sep 10 17:15:55.604459 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.604391    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-5mpq7" podStartSLOduration=1.17989663 podStartE2EDuration="17.604368027s" totalImagesPullingTime="16.424471397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:15:55.603067721 +0000 UTC m=+159.284240665" watchObservedRunningTime="2026-09-10 17:15:55.604368027 +0000 UTC m=+159.285540972"
Sep 10 17:15:55.777864 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:55.777714    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78fc566c97-xwzzz"]
Sep 10 17:15:55.780534 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:15:55.780495    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cab00f2_5dfa_43d9_b45f_96d2f79d1811.slice/crio-66b3d40ff310feaa334aa20e9435877c90493f8d4195e166370b2c8416ca4c6a WatchSource:0}: Error finding container 66b3d40ff310feaa334aa20e9435877c90493f8d4195e166370b2c8416ca4c6a: Status 404 returned error can't find the container with id 66b3d40ff310feaa334aa20e9435877c90493f8d4195e166370b2c8416ca4c6a
Sep 10 17:15:56.193341 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:56.193185    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:56.593200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:56.593154    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78fc566c97-xwzzz" event={"ID":"1cab00f2-5dfa-43d9-b45f-96d2f79d1811","Type":"ContainerStarted","Data":"66b3d40ff310feaa334aa20e9435877c90493f8d4195e166370b2c8416ca4c6a"}
Sep 10 17:15:57.128828 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:57.128728    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vkbjg_7cf62bc7-5bc1-4f6a-b73b-a7e488ab8c6f/serve-healthcheck-canary/0.log"
Sep 10 17:15:58.579518 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:58.578961    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:15:58.582451 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:58.582377    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f11bf9c-45a3-4b27-8f8d-4a1d52999da2-metrics-tls\") pod \"dns-default-pwhzb\" (UID: \"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2\") " pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:15:58.775534 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:58.775399    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2rm4x\""
Sep 10 17:15:58.783163 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:58.783060    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:15:58.994240 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:58.993994    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:59.056662 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:59.056611    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:15:59.416418 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:59.416387    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pwhzb"]
Sep 10 17:15:59.543771 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:15:59.543730    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f11bf9c_45a3_4b27_8f8d_4a1d52999da2.slice/crio-bef3cfec48121c24de761b8e6626ec3b8ef86773ee90bbf2cdd7e818014391e5 WatchSource:0}: Error finding container bef3cfec48121c24de761b8e6626ec3b8ef86773ee90bbf2cdd7e818014391e5: Status 404 returned error can't find the container with id bef3cfec48121c24de761b8e6626ec3b8ef86773ee90bbf2cdd7e818014391e5
Sep 10 17:15:59.614339 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:15:59.614297    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pwhzb" event={"ID":"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2","Type":"ContainerStarted","Data":"bef3cfec48121c24de761b8e6626ec3b8ef86773ee90bbf2cdd7e818014391e5"}
Sep 10 17:16:00.623537 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:00.623495    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d44b69d6-v8tm8" event={"ID":"3ae8d927-c657-4f1a-83ab-a4e596ec21bc","Type":"ContainerStarted","Data":"3c3f6df5ce063134684f1b4a0796ee7a53b51137c4ceca25cf8767ad17bf2c9b"}
Sep 10 17:16:00.626700 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:00.626662    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78fc566c97-xwzzz" event={"ID":"1cab00f2-5dfa-43d9-b45f-96d2f79d1811","Type":"ContainerStarted","Data":"5246a4fda22c85fc184898b5492292b05eeffcabed45d67e5899d5adc1c01d0d"}
Sep 10 17:16:00.643894 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:00.643816    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57d44b69d6-v8tm8" podStartSLOduration=8.413735646 podStartE2EDuration="12.643792799s" totalImagesPullingTime="4.230057153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:15:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:16:00.641974472 +0000 UTC m=+164.323147454" watchObservedRunningTime="2026-09-10 17:16:00.643792799 +0000 UTC m=+164.324965757"
Sep 10 17:16:00.663194 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:00.663085    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78fc566c97-xwzzz" podStartSLOduration=1.722603001 podStartE2EDuration="5.663061682s" totalImagesPullingTime="3.940458681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:15:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:16:00.662164586 +0000 UTC m=+164.343337530" watchObservedRunningTime="2026-09-10 17:16:00.663061682 +0000 UTC m=+164.344234628"
Sep 10 17:16:01.398971 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:01.398929    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:01.462783 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:01.462730    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:02.638344 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:02.638297    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pwhzb" event={"ID":"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2","Type":"ContainerStarted","Data":"20cda5c3ec60313f50b76401cf9e357e4a8f4a0cfb26dc447dd46f92c6121857"}
Sep 10 17:16:02.638856 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:02.638372    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pwhzb" event={"ID":"2f11bf9c-45a3-4b27-8f8d-4a1d52999da2","Type":"ContainerStarted","Data":"9a945a32d1ac94af43d8e2b71e3e988a492cf2d71326bc9f3a20e2276bce2039"}
Sep 10 17:16:02.638856 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:02.638445    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:16:02.659896 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:02.659838    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pwhzb" podStartSLOduration=130.367037079 podStartE2EDuration="2m12.65981839s" totalImagesPullingTime="2.292781311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:13:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:16:02.657521088 +0000 UTC m=+166.338694048" watchObservedRunningTime="2026-09-10 17:16:02.65981839 +0000 UTC m=+166.340991333"
Sep 10 17:16:02.831021 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:02.830963    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:02.915001 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:02.914901    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:05.493644 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:05.493592    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:16:05.494152 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:05.493659    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:16:05.498743 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:05.498718    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:16:05.657614 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:05.657586    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:16:05.717304 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:05.715563    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57d44b69d6-v8tm8"]
Sep 10 17:16:05.833684 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:05.833561    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:05.935398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:05.935352    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:06.003823 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:06.003786    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:06.547255 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:06.547189    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:06.617761 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:06.617705    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:08.313098 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:08.313051    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:08.416162 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:08.416109    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:08.567048 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:08.566958    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:16:12.646917 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:12.646887    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pwhzb"
Sep 10 17:16:16.687842 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:16.687801    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:16.909966 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:16.909927    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:17.010684 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:17.010643    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:17.110732 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:17.110693    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:21.803577 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:21.803538    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:21.866339 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:21.866299    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:23.301770 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:23.301729    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:23.401672 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:23.401633    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:30.737652 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:30.737587    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-57d44b69d6-v8tm8" podUID="3ae8d927-c657-4f1a-83ab-a4e596ec21bc" containerName="console" containerID="cri-o://3c3f6df5ce063134684f1b4a0796ee7a53b51137c4ceca25cf8767ad17bf2c9b" gracePeriod=15
Sep 10 17:16:31.031228 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.031192    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:16:31.172949 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.172910    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-oauth-serving-cert\") pod \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") "
Sep 10 17:16:31.173147 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.172968    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-console-serving-cert\") pod \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") "
Sep 10 17:16:31.173147 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.173024    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4kfz\" (UniqueName: \"kubernetes.io/projected/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-kube-api-access-k4kfz\") pod \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") "
Sep 10 17:16:31.173147 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.173041    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-service-ca\") pod \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") "
Sep 10 17:16:31.173147 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.173080    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-console-oauth-config\") pod \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") "
Sep 10 17:16:31.173352 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.173200    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-console-config\") pod \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\" (UID: \"3ae8d927-c657-4f1a-83ab-a4e596ec21bc\") "
Sep 10 17:16:31.173551 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.173518    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-service-ca" (OuterVolumeSpecName: "service-ca") pod "3ae8d927-c657-4f1a-83ab-a4e596ec21bc" (UID: "3ae8d927-c657-4f1a-83ab-a4e596ec21bc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:16:31.173661 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.173430    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3ae8d927-c657-4f1a-83ab-a4e596ec21bc" (UID: "3ae8d927-c657-4f1a-83ab-a4e596ec21bc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:16:31.173661 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.173593    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-console-config" (OuterVolumeSpecName: "console-config") pod "3ae8d927-c657-4f1a-83ab-a4e596ec21bc" (UID: "3ae8d927-c657-4f1a-83ab-a4e596ec21bc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:16:31.175594 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.175562    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3ae8d927-c657-4f1a-83ab-a4e596ec21bc" (UID: "3ae8d927-c657-4f1a-83ab-a4e596ec21bc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:16:31.175594 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.175566    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-kube-api-access-k4kfz" (OuterVolumeSpecName: "kube-api-access-k4kfz") pod "3ae8d927-c657-4f1a-83ab-a4e596ec21bc" (UID: "3ae8d927-c657-4f1a-83ab-a4e596ec21bc"). InnerVolumeSpecName "kube-api-access-k4kfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:16:31.175732 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.175600    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3ae8d927-c657-4f1a-83ab-a4e596ec21bc" (UID: "3ae8d927-c657-4f1a-83ab-a4e596ec21bc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:16:31.274061 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.274024    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-oauth-serving-cert\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:31.274061 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.274056    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-console-serving-cert\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:31.274061 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.274065    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4kfz\" (UniqueName: \"kubernetes.io/projected/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-kube-api-access-k4kfz\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:31.274305 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.274075    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:31.274305 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.274083    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-console-oauth-config\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:31.274305 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.274092    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ae8d927-c657-4f1a-83ab-a4e596ec21bc-console-config\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:31.764649 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.764603    2566 generic.go:356] "Generic (PLEG): container finished" podID="3ae8d927-c657-4f1a-83ab-a4e596ec21bc" containerID="3c3f6df5ce063134684f1b4a0796ee7a53b51137c4ceca25cf8767ad17bf2c9b" exitCode=0
Sep 10 17:16:31.765077 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.764661    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d44b69d6-v8tm8"
Sep 10 17:16:31.765077 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.764680    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d44b69d6-v8tm8" event={"ID":"3ae8d927-c657-4f1a-83ab-a4e596ec21bc","Type":"ContainerDied","Data":"3c3f6df5ce063134684f1b4a0796ee7a53b51137c4ceca25cf8767ad17bf2c9b"}
Sep 10 17:16:31.765077 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.764762    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d44b69d6-v8tm8" event={"ID":"3ae8d927-c657-4f1a-83ab-a4e596ec21bc","Type":"ContainerDied","Data":"6f4891dfd773f1217eeeb79eb39ae017a8b07541b46828ac221643fc2c8bc72c"}
Sep 10 17:16:31.765077 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.764784    2566 scope.go:117] "RemoveContainer" containerID="3c3f6df5ce063134684f1b4a0796ee7a53b51137c4ceca25cf8767ad17bf2c9b"
Sep 10 17:16:31.774289 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.774266    2566 scope.go:117] "RemoveContainer" containerID="3c3f6df5ce063134684f1b4a0796ee7a53b51137c4ceca25cf8767ad17bf2c9b"
Sep 10 17:16:31.774625 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:16:31.774602    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c3f6df5ce063134684f1b4a0796ee7a53b51137c4ceca25cf8767ad17bf2c9b\": container with ID starting with 3c3f6df5ce063134684f1b4a0796ee7a53b51137c4ceca25cf8767ad17bf2c9b not found: ID does not exist" containerID="3c3f6df5ce063134684f1b4a0796ee7a53b51137c4ceca25cf8767ad17bf2c9b"
Sep 10 17:16:31.774669 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.774634    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c3f6df5ce063134684f1b4a0796ee7a53b51137c4ceca25cf8767ad17bf2c9b"} err="failed to get container status \"3c3f6df5ce063134684f1b4a0796ee7a53b51137c4ceca25cf8767ad17bf2c9b\": rpc error: code = NotFound desc = could not find container \"3c3f6df5ce063134684f1b4a0796ee7a53b51137c4ceca25cf8767ad17bf2c9b\": container with ID starting with 3c3f6df5ce063134684f1b4a0796ee7a53b51137c4ceca25cf8767ad17bf2c9b not found: ID does not exist"
Sep 10 17:16:31.786209 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.786174    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57d44b69d6-v8tm8"]
Sep 10 17:16:31.792050 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:31.792019    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57d44b69d6-v8tm8"]
Sep 10 17:16:32.833555 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:32.833522    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ae8d927-c657-4f1a-83ab-a4e596ec21bc" path="/var/lib/kubelet/pods/3ae8d927-c657-4f1a-83ab-a4e596ec21bc/volumes"
Sep 10 17:16:44.698599 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:44.698554    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:45.132408 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:45.132366    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:45.749733 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:45.749697    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 17:16:45.750165 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:45.750025    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="alertmanager" containerID="cri-o://fb082b4bb45e1082f08bda14f6a96f24f738a13e7717c20571dc88d8da99771a" gracePeriod=120
Sep 10 17:16:45.750165 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:45.750070    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="kube-rbac-proxy-web" containerID="cri-o://0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3" gracePeriod=120
Sep 10 17:16:45.750165 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:45.750096    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="kube-rbac-proxy" containerID="cri-o://86888fbea2e780249d15627ae6dc0e72673fb251f18cf2254b521029f4246a68" gracePeriod=120
Sep 10 17:16:45.750336 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:45.750120    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="config-reloader" containerID="cri-o://f0c5bbe03139619cdf3887d81fafbf20d78ed5d2c414c3f63d694a419c4db00e" gracePeriod=120
Sep 10 17:16:45.750336 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:45.750163    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="prom-label-proxy" containerID="cri-o://837d25db185e428ca6a7f45b7232e39a85b33e4dd8b4cd04a3dddb26bfbc7306" gracePeriod=120
Sep 10 17:16:45.750336 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:45.750070    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="kube-rbac-proxy-metric" containerID="cri-o://4c92ffcbe8757554760f9a8960004819c68c6477767754befbefaae5531b01e8" gracePeriod=120
Sep 10 17:16:46.835653 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:46.835614    2566 generic.go:356] "Generic (PLEG): container finished" podID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerID="837d25db185e428ca6a7f45b7232e39a85b33e4dd8b4cd04a3dddb26bfbc7306" exitCode=0
Sep 10 17:16:46.835653 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:46.835645    2566 generic.go:356] "Generic (PLEG): container finished" podID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerID="86888fbea2e780249d15627ae6dc0e72673fb251f18cf2254b521029f4246a68" exitCode=0
Sep 10 17:16:46.835653 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:46.835654    2566 generic.go:356] "Generic (PLEG): container finished" podID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerID="f0c5bbe03139619cdf3887d81fafbf20d78ed5d2c414c3f63d694a419c4db00e" exitCode=0
Sep 10 17:16:46.835653 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:46.835663    2566 generic.go:356] "Generic (PLEG): container finished" podID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerID="fb082b4bb45e1082f08bda14f6a96f24f738a13e7717c20571dc88d8da99771a" exitCode=0
Sep 10 17:16:46.836484 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:46.835681    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"78250d22-60e4-45ec-b893-2edd9dd8a390","Type":"ContainerDied","Data":"837d25db185e428ca6a7f45b7232e39a85b33e4dd8b4cd04a3dddb26bfbc7306"}
Sep 10 17:16:46.836484 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:46.835761    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"78250d22-60e4-45ec-b893-2edd9dd8a390","Type":"ContainerDied","Data":"86888fbea2e780249d15627ae6dc0e72673fb251f18cf2254b521029f4246a68"}
Sep 10 17:16:46.836484 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:46.835802    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"78250d22-60e4-45ec-b893-2edd9dd8a390","Type":"ContainerDied","Data":"f0c5bbe03139619cdf3887d81fafbf20d78ed5d2c414c3f63d694a419c4db00e"}
Sep 10 17:16:46.836484 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:46.835835    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"78250d22-60e4-45ec-b893-2edd9dd8a390","Type":"ContainerDied","Data":"fb082b4bb45e1082f08bda14f6a96f24f738a13e7717c20571dc88d8da99771a"}
Sep 10 17:16:46.836792 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:16:46.836770    2566 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78250d22_60e4_45ec_b893_2edd9dd8a390.slice/crio-0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 17:16:47.013196 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.013166    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:47.105014 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.104929    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-config-volume\") pod \"78250d22-60e4-45ec-b893-2edd9dd8a390\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") "
Sep 10 17:16:47.105207 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.105025    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/78250d22-60e4-45ec-b893-2edd9dd8a390-tls-assets\") pod \"78250d22-60e4-45ec-b893-2edd9dd8a390\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") "
Sep 10 17:16:47.105207 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.105054    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-main-tls\") pod \"78250d22-60e4-45ec-b893-2edd9dd8a390\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") "
Sep 10 17:16:47.105207 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.105100    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78250d22-60e4-45ec-b893-2edd9dd8a390-alertmanager-trusted-ca-bundle\") pod \"78250d22-60e4-45ec-b893-2edd9dd8a390\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") "
Sep 10 17:16:47.105207 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.105190    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78250d22-60e4-45ec-b893-2edd9dd8a390-metrics-client-ca\") pod \"78250d22-60e4-45ec-b893-2edd9dd8a390\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") "
Sep 10 17:16:47.105426 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.105228    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy-web\") pod \"78250d22-60e4-45ec-b893-2edd9dd8a390\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") "
Sep 10 17:16:47.105426 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.105259    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-web-config\") pod \"78250d22-60e4-45ec-b893-2edd9dd8a390\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") "
Sep 10 17:16:47.105426 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.105285    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xf9rr\" (UniqueName: \"kubernetes.io/projected/78250d22-60e4-45ec-b893-2edd9dd8a390-kube-api-access-xf9rr\") pod \"78250d22-60e4-45ec-b893-2edd9dd8a390\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") "
Sep 10 17:16:47.105426 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.105327    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/78250d22-60e4-45ec-b893-2edd9dd8a390-alertmanager-main-db\") pod \"78250d22-60e4-45ec-b893-2edd9dd8a390\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") "
Sep 10 17:16:47.105426 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.105378    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy\") pod \"78250d22-60e4-45ec-b893-2edd9dd8a390\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") "
Sep 10 17:16:47.105426 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.105406    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-cluster-tls-config\") pod \"78250d22-60e4-45ec-b893-2edd9dd8a390\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") "
Sep 10 17:16:47.105788 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.105433    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy-metric\") pod \"78250d22-60e4-45ec-b893-2edd9dd8a390\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") "
Sep 10 17:16:47.105788 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.105586    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78250d22-60e4-45ec-b893-2edd9dd8a390-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "78250d22-60e4-45ec-b893-2edd9dd8a390" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:16:47.105788 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.105681    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78250d22-60e4-45ec-b893-2edd9dd8a390-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "78250d22-60e4-45ec-b893-2edd9dd8a390" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:16:47.105788 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.105713    2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78250d22-60e4-45ec-b893-2edd9dd8a390-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:47.106277 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.106116    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78250d22-60e4-45ec-b893-2edd9dd8a390-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "78250d22-60e4-45ec-b893-2edd9dd8a390" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:16:47.108552 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.108398    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "78250d22-60e4-45ec-b893-2edd9dd8a390" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:16:47.108552 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.108495    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-config-volume" (OuterVolumeSpecName: "config-volume") pod "78250d22-60e4-45ec-b893-2edd9dd8a390" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:16:47.108722 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.108679    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "78250d22-60e4-45ec-b893-2edd9dd8a390" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:16:47.108904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.108878    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78250d22-60e4-45ec-b893-2edd9dd8a390-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "78250d22-60e4-45ec-b893-2edd9dd8a390" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:16:47.109078 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.109054    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "78250d22-60e4-45ec-b893-2edd9dd8a390" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:16:47.109402 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.109371    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "78250d22-60e4-45ec-b893-2edd9dd8a390" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:16:47.109694 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.109665    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78250d22-60e4-45ec-b893-2edd9dd8a390-kube-api-access-xf9rr" (OuterVolumeSpecName: "kube-api-access-xf9rr") pod "78250d22-60e4-45ec-b893-2edd9dd8a390" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390"). InnerVolumeSpecName "kube-api-access-xf9rr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:16:47.113785 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.113751    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "78250d22-60e4-45ec-b893-2edd9dd8a390" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:16:47.120448 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.120410    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-web-config" (OuterVolumeSpecName: "web-config") pod "78250d22-60e4-45ec-b893-2edd9dd8a390" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:16:47.206157 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.206091    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/78250d22-60e4-45ec-b893-2edd9dd8a390-config-out\") pod \"78250d22-60e4-45ec-b893-2edd9dd8a390\" (UID: \"78250d22-60e4-45ec-b893-2edd9dd8a390\") "
Sep 10 17:16:47.206439 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.206415    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78250d22-60e4-45ec-b893-2edd9dd8a390-metrics-client-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:47.206486 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.206448    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:47.206486 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.206466    2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-web-config\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:47.206486 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.206479    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xf9rr\" (UniqueName: \"kubernetes.io/projected/78250d22-60e4-45ec-b893-2edd9dd8a390-kube-api-access-xf9rr\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:47.206579 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.206492    2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/78250d22-60e4-45ec-b893-2edd9dd8a390-alertmanager-main-db\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:47.206579 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.206505    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:47.206579 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.206516    2566 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-cluster-tls-config\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:47.206579 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.206525    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:47.206579 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.206534    2566 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-config-volume\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:47.206579 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.206542    2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/78250d22-60e4-45ec-b893-2edd9dd8a390-tls-assets\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:47.206579 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.206550    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/78250d22-60e4-45ec-b893-2edd9dd8a390-secret-alertmanager-main-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:47.208267 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.208238    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78250d22-60e4-45ec-b893-2edd9dd8a390-config-out" (OuterVolumeSpecName: "config-out") pod "78250d22-60e4-45ec-b893-2edd9dd8a390" (UID: "78250d22-60e4-45ec-b893-2edd9dd8a390"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:16:47.307107 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.307059    2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/78250d22-60e4-45ec-b893-2edd9dd8a390-config-out\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:16:47.843554 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.843520    2566 generic.go:356] "Generic (PLEG): container finished" podID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerID="4c92ffcbe8757554760f9a8960004819c68c6477767754befbefaae5531b01e8" exitCode=0
Sep 10 17:16:47.843554 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.843548    2566 generic.go:356] "Generic (PLEG): container finished" podID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerID="0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3" exitCode=0
Sep 10 17:16:47.844099 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.843612    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"78250d22-60e4-45ec-b893-2edd9dd8a390","Type":"ContainerDied","Data":"4c92ffcbe8757554760f9a8960004819c68c6477767754befbefaae5531b01e8"}
Sep 10 17:16:47.844099 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.843639    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:47.844099 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.843689    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"78250d22-60e4-45ec-b893-2edd9dd8a390","Type":"ContainerDied","Data":"0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3"}
Sep 10 17:16:47.844099 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.843719    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"78250d22-60e4-45ec-b893-2edd9dd8a390","Type":"ContainerDied","Data":"e26cb21fc5b8dc3c22de2114088785686489c29a58f0cb938d14064347562a7f"}
Sep 10 17:16:47.844099 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.843757    2566 scope.go:117] "RemoveContainer" containerID="837d25db185e428ca6a7f45b7232e39a85b33e4dd8b4cd04a3dddb26bfbc7306"
Sep 10 17:16:47.851946 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.851910    2566 scope.go:117] "RemoveContainer" containerID="4c92ffcbe8757554760f9a8960004819c68c6477767754befbefaae5531b01e8"
Sep 10 17:16:47.859750 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.859728    2566 scope.go:117] "RemoveContainer" containerID="86888fbea2e780249d15627ae6dc0e72673fb251f18cf2254b521029f4246a68"
Sep 10 17:16:47.867620 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.867592    2566 scope.go:117] "RemoveContainer" containerID="0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3"
Sep 10 17:16:47.869462 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.869434    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 17:16:47.875057 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.875023    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 17:16:47.877540 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.877457    2566 scope.go:117] "RemoveContainer" containerID="f0c5bbe03139619cdf3887d81fafbf20d78ed5d2c414c3f63d694a419c4db00e"
Sep 10 17:16:47.888059 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.888038    2566 scope.go:117] "RemoveContainer" containerID="fb082b4bb45e1082f08bda14f6a96f24f738a13e7717c20571dc88d8da99771a"
Sep 10 17:16:47.895920 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.895894    2566 scope.go:117] "RemoveContainer" containerID="3dd6db0af04abf78c709262d6be0563b41423ea141e58eefeb63050214519e80"
Sep 10 17:16:47.903730 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.903697    2566 scope.go:117] "RemoveContainer" containerID="837d25db185e428ca6a7f45b7232e39a85b33e4dd8b4cd04a3dddb26bfbc7306"
Sep 10 17:16:47.904049 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:16:47.904028    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"837d25db185e428ca6a7f45b7232e39a85b33e4dd8b4cd04a3dddb26bfbc7306\": container with ID starting with 837d25db185e428ca6a7f45b7232e39a85b33e4dd8b4cd04a3dddb26bfbc7306 not found: ID does not exist" containerID="837d25db185e428ca6a7f45b7232e39a85b33e4dd8b4cd04a3dddb26bfbc7306"
Sep 10 17:16:47.904099 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.904060    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"837d25db185e428ca6a7f45b7232e39a85b33e4dd8b4cd04a3dddb26bfbc7306"} err="failed to get container status \"837d25db185e428ca6a7f45b7232e39a85b33e4dd8b4cd04a3dddb26bfbc7306\": rpc error: code = NotFound desc = could not find container \"837d25db185e428ca6a7f45b7232e39a85b33e4dd8b4cd04a3dddb26bfbc7306\": container with ID starting with 837d25db185e428ca6a7f45b7232e39a85b33e4dd8b4cd04a3dddb26bfbc7306 not found: ID does not exist"
Sep 10 17:16:47.904099 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.904081    2566 scope.go:117] "RemoveContainer" containerID="4c92ffcbe8757554760f9a8960004819c68c6477767754befbefaae5531b01e8"
Sep 10 17:16:47.904466 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:16:47.904445    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c92ffcbe8757554760f9a8960004819c68c6477767754befbefaae5531b01e8\": container with ID starting with 4c92ffcbe8757554760f9a8960004819c68c6477767754befbefaae5531b01e8 not found: ID does not exist" containerID="4c92ffcbe8757554760f9a8960004819c68c6477767754befbefaae5531b01e8"
Sep 10 17:16:47.904537 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.904472    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c92ffcbe8757554760f9a8960004819c68c6477767754befbefaae5531b01e8"} err="failed to get container status \"4c92ffcbe8757554760f9a8960004819c68c6477767754befbefaae5531b01e8\": rpc error: code = NotFound desc = could not find container \"4c92ffcbe8757554760f9a8960004819c68c6477767754befbefaae5531b01e8\": container with ID starting with 4c92ffcbe8757554760f9a8960004819c68c6477767754befbefaae5531b01e8 not found: ID does not exist"
Sep 10 17:16:47.904537 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.904490    2566 scope.go:117] "RemoveContainer" containerID="86888fbea2e780249d15627ae6dc0e72673fb251f18cf2254b521029f4246a68"
Sep 10 17:16:47.904768 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:16:47.904752    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86888fbea2e780249d15627ae6dc0e72673fb251f18cf2254b521029f4246a68\": container with ID starting with 86888fbea2e780249d15627ae6dc0e72673fb251f18cf2254b521029f4246a68 not found: ID does not exist" containerID="86888fbea2e780249d15627ae6dc0e72673fb251f18cf2254b521029f4246a68"
Sep 10 17:16:47.904804 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.904775    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86888fbea2e780249d15627ae6dc0e72673fb251f18cf2254b521029f4246a68"} err="failed to get container status \"86888fbea2e780249d15627ae6dc0e72673fb251f18cf2254b521029f4246a68\": rpc error: code = NotFound desc = could not find container \"86888fbea2e780249d15627ae6dc0e72673fb251f18cf2254b521029f4246a68\": container with ID starting with 86888fbea2e780249d15627ae6dc0e72673fb251f18cf2254b521029f4246a68 not found: ID does not exist"
Sep 10 17:16:47.904804 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.904794    2566 scope.go:117] "RemoveContainer" containerID="0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3"
Sep 10 17:16:47.905040 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:16:47.905024    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3\": container with ID starting with 0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3 not found: ID does not exist" containerID="0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3"
Sep 10 17:16:47.905085 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.905045    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3"} err="failed to get container status \"0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3\": rpc error: code = NotFound desc = could not find container \"0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3\": container with ID starting with 0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3 not found: ID does not exist"
Sep 10 17:16:47.905085 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.905061    2566 scope.go:117] "RemoveContainer" containerID="f0c5bbe03139619cdf3887d81fafbf20d78ed5d2c414c3f63d694a419c4db00e"
Sep 10 17:16:47.905311 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:16:47.905295    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0c5bbe03139619cdf3887d81fafbf20d78ed5d2c414c3f63d694a419c4db00e\": container with ID starting with f0c5bbe03139619cdf3887d81fafbf20d78ed5d2c414c3f63d694a419c4db00e not found: ID does not exist" containerID="f0c5bbe03139619cdf3887d81fafbf20d78ed5d2c414c3f63d694a419c4db00e"
Sep 10 17:16:47.905354 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.905314    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0c5bbe03139619cdf3887d81fafbf20d78ed5d2c414c3f63d694a419c4db00e"} err="failed to get container status \"f0c5bbe03139619cdf3887d81fafbf20d78ed5d2c414c3f63d694a419c4db00e\": rpc error: code = NotFound desc = could not find container \"f0c5bbe03139619cdf3887d81fafbf20d78ed5d2c414c3f63d694a419c4db00e\": container with ID starting with f0c5bbe03139619cdf3887d81fafbf20d78ed5d2c414c3f63d694a419c4db00e not found: ID does not exist"
Sep 10 17:16:47.905354 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.905327    2566 scope.go:117] "RemoveContainer" containerID="fb082b4bb45e1082f08bda14f6a96f24f738a13e7717c20571dc88d8da99771a"
Sep 10 17:16:47.905567 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:16:47.905551    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb082b4bb45e1082f08bda14f6a96f24f738a13e7717c20571dc88d8da99771a\": container with ID starting with fb082b4bb45e1082f08bda14f6a96f24f738a13e7717c20571dc88d8da99771a not found: ID does not exist" containerID="fb082b4bb45e1082f08bda14f6a96f24f738a13e7717c20571dc88d8da99771a"
Sep 10 17:16:47.905616 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.905569    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb082b4bb45e1082f08bda14f6a96f24f738a13e7717c20571dc88d8da99771a"} err="failed to get container status \"fb082b4bb45e1082f08bda14f6a96f24f738a13e7717c20571dc88d8da99771a\": rpc error: code = NotFound desc = could not find container \"fb082b4bb45e1082f08bda14f6a96f24f738a13e7717c20571dc88d8da99771a\": container with ID starting with fb082b4bb45e1082f08bda14f6a96f24f738a13e7717c20571dc88d8da99771a not found: ID does not exist"
Sep 10 17:16:47.905616 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.905582    2566 scope.go:117] "RemoveContainer" containerID="3dd6db0af04abf78c709262d6be0563b41423ea141e58eefeb63050214519e80"
Sep 10 17:16:47.905807 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:16:47.905783    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dd6db0af04abf78c709262d6be0563b41423ea141e58eefeb63050214519e80\": container with ID starting with 3dd6db0af04abf78c709262d6be0563b41423ea141e58eefeb63050214519e80 not found: ID does not exist" containerID="3dd6db0af04abf78c709262d6be0563b41423ea141e58eefeb63050214519e80"
Sep 10 17:16:47.905849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.905813    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dd6db0af04abf78c709262d6be0563b41423ea141e58eefeb63050214519e80"} err="failed to get container status \"3dd6db0af04abf78c709262d6be0563b41423ea141e58eefeb63050214519e80\": rpc error: code = NotFound desc = could not find container \"3dd6db0af04abf78c709262d6be0563b41423ea141e58eefeb63050214519e80\": container with ID starting with 3dd6db0af04abf78c709262d6be0563b41423ea141e58eefeb63050214519e80 not found: ID does not exist"
Sep 10 17:16:47.905849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.905830    2566 scope.go:117] "RemoveContainer" containerID="837d25db185e428ca6a7f45b7232e39a85b33e4dd8b4cd04a3dddb26bfbc7306"
Sep 10 17:16:47.906056 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.906039    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"837d25db185e428ca6a7f45b7232e39a85b33e4dd8b4cd04a3dddb26bfbc7306"} err="failed to get container status \"837d25db185e428ca6a7f45b7232e39a85b33e4dd8b4cd04a3dddb26bfbc7306\": rpc error: code = NotFound desc = could not find container \"837d25db185e428ca6a7f45b7232e39a85b33e4dd8b4cd04a3dddb26bfbc7306\": container with ID starting with 837d25db185e428ca6a7f45b7232e39a85b33e4dd8b4cd04a3dddb26bfbc7306 not found: ID does not exist"
Sep 10 17:16:47.906109 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.906056    2566 scope.go:117] "RemoveContainer" containerID="4c92ffcbe8757554760f9a8960004819c68c6477767754befbefaae5531b01e8"
Sep 10 17:16:47.906352 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.906333    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c92ffcbe8757554760f9a8960004819c68c6477767754befbefaae5531b01e8"} err="failed to get container status \"4c92ffcbe8757554760f9a8960004819c68c6477767754befbefaae5531b01e8\": rpc error: code = NotFound desc = could not find container \"4c92ffcbe8757554760f9a8960004819c68c6477767754befbefaae5531b01e8\": container with ID starting with 4c92ffcbe8757554760f9a8960004819c68c6477767754befbefaae5531b01e8 not found: ID does not exist"
Sep 10 17:16:47.906416 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.906352    2566 scope.go:117] "RemoveContainer" containerID="86888fbea2e780249d15627ae6dc0e72673fb251f18cf2254b521029f4246a68"
Sep 10 17:16:47.906582 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.906564    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86888fbea2e780249d15627ae6dc0e72673fb251f18cf2254b521029f4246a68"} err="failed to get container status \"86888fbea2e780249d15627ae6dc0e72673fb251f18cf2254b521029f4246a68\": rpc error: code = NotFound desc = could not find container \"86888fbea2e780249d15627ae6dc0e72673fb251f18cf2254b521029f4246a68\": container with ID starting with 86888fbea2e780249d15627ae6dc0e72673fb251f18cf2254b521029f4246a68 not found: ID does not exist"
Sep 10 17:16:47.906620 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.906582    2566 scope.go:117] "RemoveContainer" containerID="0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3"
Sep 10 17:16:47.906803 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.906785    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3"} err="failed to get container status \"0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3\": rpc error: code = NotFound desc = could not find container \"0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3\": container with ID starting with 0c3095f56c1855591c048959c299fe0c86f06d93af81325883d24e7cb573f3f3 not found: ID does not exist"
Sep 10 17:16:47.906844 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.906803    2566 scope.go:117] "RemoveContainer" containerID="f0c5bbe03139619cdf3887d81fafbf20d78ed5d2c414c3f63d694a419c4db00e"
Sep 10 17:16:47.907022 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.907002    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0c5bbe03139619cdf3887d81fafbf20d78ed5d2c414c3f63d694a419c4db00e"} err="failed to get container status \"f0c5bbe03139619cdf3887d81fafbf20d78ed5d2c414c3f63d694a419c4db00e\": rpc error: code = NotFound desc = could not find container \"f0c5bbe03139619cdf3887d81fafbf20d78ed5d2c414c3f63d694a419c4db00e\": container with ID starting with f0c5bbe03139619cdf3887d81fafbf20d78ed5d2c414c3f63d694a419c4db00e not found: ID does not exist"
Sep 10 17:16:47.907066 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.907022    2566 scope.go:117] "RemoveContainer" containerID="fb082b4bb45e1082f08bda14f6a96f24f738a13e7717c20571dc88d8da99771a"
Sep 10 17:16:47.907265 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.907247    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb082b4bb45e1082f08bda14f6a96f24f738a13e7717c20571dc88d8da99771a"} err="failed to get container status \"fb082b4bb45e1082f08bda14f6a96f24f738a13e7717c20571dc88d8da99771a\": rpc error: code = NotFound desc = could not find container \"fb082b4bb45e1082f08bda14f6a96f24f738a13e7717c20571dc88d8da99771a\": container with ID starting with fb082b4bb45e1082f08bda14f6a96f24f738a13e7717c20571dc88d8da99771a not found: ID does not exist"
Sep 10 17:16:47.907317 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.907265    2566 scope.go:117] "RemoveContainer" containerID="3dd6db0af04abf78c709262d6be0563b41423ea141e58eefeb63050214519e80"
Sep 10 17:16:47.907500 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.907483    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dd6db0af04abf78c709262d6be0563b41423ea141e58eefeb63050214519e80"} err="failed to get container status \"3dd6db0af04abf78c709262d6be0563b41423ea141e58eefeb63050214519e80\": rpc error: code = NotFound desc = could not find container \"3dd6db0af04abf78c709262d6be0563b41423ea141e58eefeb63050214519e80\": container with ID starting with 3dd6db0af04abf78c709262d6be0563b41423ea141e58eefeb63050214519e80 not found: ID does not exist"
Sep 10 17:16:47.913361 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.913333    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 17:16:47.913734 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.913716    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="alertmanager"
Sep 10 17:16:47.913815 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.913737    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="alertmanager"
Sep 10 17:16:47.913815 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.913752    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="kube-rbac-proxy-metric"
Sep 10 17:16:47.913815 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.913762    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="kube-rbac-proxy-metric"
Sep 10 17:16:47.913815 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.913776    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="kube-rbac-proxy-web"
Sep 10 17:16:47.913815 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.913785    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="kube-rbac-proxy-web"
Sep 10 17:16:47.913815 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.913798    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ae8d927-c657-4f1a-83ab-a4e596ec21bc" containerName="console"
Sep 10 17:16:47.913815 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.913807    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ae8d927-c657-4f1a-83ab-a4e596ec21bc" containerName="console"
Sep 10 17:16:47.914159 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.913832    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="init-config-reloader"
Sep 10 17:16:47.914159 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.913842    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="init-config-reloader"
Sep 10 17:16:47.914159 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.913854    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="kube-rbac-proxy"
Sep 10 17:16:47.914159 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.913862    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="kube-rbac-proxy"
Sep 10 17:16:47.914159 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.913875    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="config-reloader"
Sep 10 17:16:47.914159 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.913884    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="config-reloader"
Sep 10 17:16:47.914159 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.913903    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="prom-label-proxy"
Sep 10 17:16:47.914159 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.913912    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="prom-label-proxy"
Sep 10 17:16:47.914159 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.913992    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ae8d927-c657-4f1a-83ab-a4e596ec21bc" containerName="console"
Sep 10 17:16:47.914159 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.914003    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="config-reloader"
Sep 10 17:16:47.914159 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.914014    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="kube-rbac-proxy"
Sep 10 17:16:47.914159 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.914024    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="prom-label-proxy"
Sep 10 17:16:47.914159 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.914033    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="alertmanager"
Sep 10 17:16:47.914159 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.914044    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="kube-rbac-proxy-web"
Sep 10 17:16:47.914159 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.914053    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" containerName="kube-rbac-proxy-metric"
Sep 10 17:16:47.919454 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.919428    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:47.923387 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.923364    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 17:16:47.923540 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.923394    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 17:16:47.923670 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.923656    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 17:16:47.924247 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.924226    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 17:16:47.924247 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.924240    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 17:16:47.924418 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.924345    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2whcg\""
Sep 10 17:16:47.924418 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.924400    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 17:16:47.924534 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.924445    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 17:16:47.924638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.924619    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 17:16:47.933557 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.933530    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 17:16:47.947024 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:47.946991    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 17:16:48.011650 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.011610    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.011650 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.011651    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c7a6e39-572d-432b-91cd-82f436f688c8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.011909 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.011671    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8c7a6e39-572d-432b-91cd-82f436f688c8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.011909 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.011740    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8c7a6e39-572d-432b-91cd-82f436f688c8-config-out\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.011909 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.011783    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.011909 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.011809    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c7a6e39-572d-432b-91cd-82f436f688c8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.011909 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.011843    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-config-volume\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.011909 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.011867    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.011909 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.011900    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.012279 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.011929    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-web-config\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.012279 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.011970    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.012279 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.011996    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49cj2\" (UniqueName: \"kubernetes.io/projected/8c7a6e39-572d-432b-91cd-82f436f688c8-kube-api-access-49cj2\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.012279 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.012037    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8c7a6e39-572d-432b-91cd-82f436f688c8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.113370 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.113264    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.113370 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.113315    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c7a6e39-572d-432b-91cd-82f436f688c8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.113370 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.113343    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8c7a6e39-572d-432b-91cd-82f436f688c8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.113370 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.113368    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8c7a6e39-572d-432b-91cd-82f436f688c8-config-out\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.113715 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.113408    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.113715 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.113432    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c7a6e39-572d-432b-91cd-82f436f688c8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.113715 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.113460    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-config-volume\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.113715 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.113485    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.113715 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.113541    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.113715 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.113588    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-web-config\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.113715 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.113622    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.113715 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.113645    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49cj2\" (UniqueName: \"kubernetes.io/projected/8c7a6e39-572d-432b-91cd-82f436f688c8-kube-api-access-49cj2\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.113715 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.113679    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8c7a6e39-572d-432b-91cd-82f436f688c8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.114115 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.114000    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8c7a6e39-572d-432b-91cd-82f436f688c8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.115558 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.114463    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c7a6e39-572d-432b-91cd-82f436f688c8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.115558 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.115442    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c7a6e39-572d-432b-91cd-82f436f688c8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.116817 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.116785    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.116950 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.116812    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.116950 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.116862    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8c7a6e39-572d-432b-91cd-82f436f688c8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.117373 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.117346    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8c7a6e39-572d-432b-91cd-82f436f688c8-config-out\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.118097 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.118070    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.118295 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.118271    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-config-volume\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.118548 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.118527    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.119275 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.119250    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.119401 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.119382    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8c7a6e39-572d-432b-91cd-82f436f688c8-web-config\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.127921 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.127895    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49cj2\" (UniqueName: \"kubernetes.io/projected/8c7a6e39-572d-432b-91cd-82f436f688c8-kube-api-access-49cj2\") pod \"alertmanager-main-0\" (UID: \"8c7a6e39-572d-432b-91cd-82f436f688c8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.163615 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.163575    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:48.229774 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.229732    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:16:48.524709 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.524664    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 17:16:48.528965 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:16:48.528935    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c7a6e39_572d_432b_91cd_82f436f688c8.slice/crio-5d0707175225dbef77521a48d9a7389c0cf756c1bb1b25d086be416f01758de8 WatchSource:0}: Error finding container 5d0707175225dbef77521a48d9a7389c0cf756c1bb1b25d086be416f01758de8: Status 404 returned error can't find the container with id 5d0707175225dbef77521a48d9a7389c0cf756c1bb1b25d086be416f01758de8
Sep 10 17:16:48.833960 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.833875    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78250d22-60e4-45ec-b893-2edd9dd8a390" path="/var/lib/kubelet/pods/78250d22-60e4-45ec-b893-2edd9dd8a390/volumes"
Sep 10 17:16:48.850545 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.850510    2566 generic.go:356] "Generic (PLEG): container finished" podID="8c7a6e39-572d-432b-91cd-82f436f688c8" containerID="4669ff358ef41e8ada7d7544ae331507594c31291f4e552bde5f24898ee466fb" exitCode=0
Sep 10 17:16:48.850943 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.850593    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8c7a6e39-572d-432b-91cd-82f436f688c8","Type":"ContainerDied","Data":"4669ff358ef41e8ada7d7544ae331507594c31291f4e552bde5f24898ee466fb"}
Sep 10 17:16:48.850943 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:48.850675    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8c7a6e39-572d-432b-91cd-82f436f688c8","Type":"ContainerStarted","Data":"5d0707175225dbef77521a48d9a7389c0cf756c1bb1b25d086be416f01758de8"}
Sep 10 17:16:49.797574 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.797512    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"]
Sep 10 17:16:49.801303 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.801270    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.804064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.804004    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 10 17:16:49.804294 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.804144    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 10 17:16:49.804366 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.804318    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 10 17:16:49.804417 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.804388    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-b68pm\""
Sep 10 17:16:49.804481 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.804388    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 10 17:16:49.806260 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.806237    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 10 17:16:49.817887 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.817847    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 10 17:16:49.828819 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.828785    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"]
Sep 10 17:16:49.829997 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.829974    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff3911e9-03a4-4893-877a-795506a81a05-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.830077 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.830056    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff3911e9-03a4-4893-877a-795506a81a05-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.830144 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.830083    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwml7\" (UniqueName: \"kubernetes.io/projected/ff3911e9-03a4-4893-877a-795506a81a05-kube-api-access-lwml7\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.830144 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.830115    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff3911e9-03a4-4893-877a-795506a81a05-serving-certs-ca-bundle\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.830225 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.830179    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ff3911e9-03a4-4893-877a-795506a81a05-federate-client-tls\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.830225 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.830218    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ff3911e9-03a4-4893-877a-795506a81a05-telemeter-client-tls\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.830315 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.830263    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff3911e9-03a4-4893-877a-795506a81a05-metrics-client-ca\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.830391 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.830339    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ff3911e9-03a4-4893-877a-795506a81a05-secret-telemeter-client\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.859093 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.859054    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8c7a6e39-572d-432b-91cd-82f436f688c8","Type":"ContainerStarted","Data":"df1781a8c19e486c32971ba3320f0471d49417f72c85613bf94b4be908f4c005"}
Sep 10 17:16:49.859471 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.859149    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8c7a6e39-572d-432b-91cd-82f436f688c8","Type":"ContainerStarted","Data":"be1a1e0646599ee7ccf58ca23d55c4207ae8286b1b3800840624e1bbac6122a4"}
Sep 10 17:16:49.859471 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.859180    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8c7a6e39-572d-432b-91cd-82f436f688c8","Type":"ContainerStarted","Data":"ae9ac1e491359c816eada5a60a699e7da3400b74f3057673379c012ee5692a92"}
Sep 10 17:16:49.859471 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.859212    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8c7a6e39-572d-432b-91cd-82f436f688c8","Type":"ContainerStarted","Data":"57973e960ac20de81c4a16ae7781bf075b2a055a65ae28dfb19ac7fe4336dad4"}
Sep 10 17:16:49.859471 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.859247    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8c7a6e39-572d-432b-91cd-82f436f688c8","Type":"ContainerStarted","Data":"6e13fb12282a12779e8187249cdc0097d23df1c0a07ead4941d3b16f767de4e6"}
Sep 10 17:16:49.859471 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.859281    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8c7a6e39-572d-432b-91cd-82f436f688c8","Type":"ContainerStarted","Data":"961da9ae072d94606b201d7e0460e21f9e284378d8a6df42a51b74875a129141"}
Sep 10 17:16:49.890392 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.890305    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.8751444729999998 podStartE2EDuration="2.890281426s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.136953ms" isStatefulPod=false podCreationTimestamp="2026-09-10 17:16:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:16:49.889011314 +0000 UTC m=+213.570184295" watchObservedRunningTime="2026-09-10 17:16:49.890281426 +0000 UTC m=+213.571454364"
Sep 10 17:16:49.930862 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.930820    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff3911e9-03a4-4893-877a-795506a81a05-metrics-client-ca\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.931066 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.931026    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ff3911e9-03a4-4893-877a-795506a81a05-secret-telemeter-client\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.931161 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.931088    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff3911e9-03a4-4893-877a-795506a81a05-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.931286 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.931263    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff3911e9-03a4-4893-877a-795506a81a05-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.931346 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.931301    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwml7\" (UniqueName: \"kubernetes.io/projected/ff3911e9-03a4-4893-877a-795506a81a05-kube-api-access-lwml7\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.931658 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.931634    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff3911e9-03a4-4893-877a-795506a81a05-metrics-client-ca\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.931782 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.931663    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff3911e9-03a4-4893-877a-795506a81a05-serving-certs-ca-bundle\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.931880 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.931859    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ff3911e9-03a4-4893-877a-795506a81a05-federate-client-tls\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.931967 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.931932    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ff3911e9-03a4-4893-877a-795506a81a05-telemeter-client-tls\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.932320 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.932215    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff3911e9-03a4-4893-877a-795506a81a05-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.933162 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.933117    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff3911e9-03a4-4893-877a-795506a81a05-serving-certs-ca-bundle\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.934677 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.934627    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ff3911e9-03a4-4893-877a-795506a81a05-federate-client-tls\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.934677 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.934654    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff3911e9-03a4-4893-877a-795506a81a05-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.935023 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.934979    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ff3911e9-03a4-4893-877a-795506a81a05-secret-telemeter-client\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.935423 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.935396    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ff3911e9-03a4-4893-877a-795506a81a05-telemeter-client-tls\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:49.946745 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:49.944856    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwml7\" (UniqueName: \"kubernetes.io/projected/ff3911e9-03a4-4893-877a-795506a81a05-kube-api-access-lwml7\") pod \"telemeter-client-7dff7cbc5-2fjcw\" (UID: \"ff3911e9-03a4-4893-877a-795506a81a05\") " pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:50.123214 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:50.122833    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"
Sep 10 17:16:50.416603 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:50.416571    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw"]
Sep 10 17:16:50.419088 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:16:50.419050    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff3911e9_03a4_4893_877a_795506a81a05.slice/crio-1a1209749f600af92531984ff51dcce4dfcf14744cd598c948deef0224054c60 WatchSource:0}: Error finding container 1a1209749f600af92531984ff51dcce4dfcf14744cd598c948deef0224054c60: Status 404 returned error can't find the container with id 1a1209749f600af92531984ff51dcce4dfcf14744cd598c948deef0224054c60
Sep 10 17:16:50.864911 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:50.864871    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw" event={"ID":"ff3911e9-03a4-4893-877a-795506a81a05","Type":"ContainerStarted","Data":"1a1209749f600af92531984ff51dcce4dfcf14744cd598c948deef0224054c60"}
Sep 10 17:16:52.876739 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:52.876700    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw" event={"ID":"ff3911e9-03a4-4893-877a-795506a81a05","Type":"ContainerStarted","Data":"3cae540c9817d1d4f2fa0a59696dd1c27777e19e885ab0da5066f0edd1ab8a34"}
Sep 10 17:16:52.877166 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:52.876764    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw" event={"ID":"ff3911e9-03a4-4893-877a-795506a81a05","Type":"ContainerStarted","Data":"d829cf7dd5a330f6b74d93b39ad1f20c65983b5659ec220734004923f0fb409a"}
Sep 10 17:16:52.877166 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:52.876784    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw" event={"ID":"ff3911e9-03a4-4893-877a-795506a81a05","Type":"ContainerStarted","Data":"ed793da755d7807a0a3dee69d8df6df7f5e41e137266912fc20315e12976679a"}
Sep 10 17:16:52.903273 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:52.903209    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7dff7cbc5-2fjcw" podStartSLOduration=2.089542802 podStartE2EDuration="3.903188629s" totalImagesPullingTime="1.813645827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:16:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:16:52.902611286 +0000 UTC m=+216.583784240" watchObservedRunningTime="2026-09-10 17:16:52.903188629 +0000 UTC m=+216.584361626"
Sep 10 17:16:53.044725 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.044685    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b4bcf79dc-4k8kt"]
Sep 10 17:16:53.048264 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.048246    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.060817 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.060786    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-service-ca\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.060953 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.060831    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56e18a61-b310-4d01-a7b4-ff1b09e11894-console-oauth-config\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.060953 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.060848    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-trusted-ca-bundle\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.060953 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.060866    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-oauth-serving-cert\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.060953 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.060883    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56e18a61-b310-4d01-a7b4-ff1b09e11894-console-serving-cert\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.061185 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.060979    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-console-config\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.061185 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.061045    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5hsd\" (UniqueName: \"kubernetes.io/projected/56e18a61-b310-4d01-a7b4-ff1b09e11894-kube-api-access-h5hsd\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.064210 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.064180    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b4bcf79dc-4k8kt"]
Sep 10 17:16:53.113525 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.113480    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:53.161723 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.161614    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56e18a61-b310-4d01-a7b4-ff1b09e11894-console-oauth-config\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.161723 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.161669    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-trusted-ca-bundle\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.161723 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.161690    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-oauth-serving-cert\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.161723 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.161707    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56e18a61-b310-4d01-a7b4-ff1b09e11894-console-serving-cert\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.162074 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.161843    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-console-config\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.162074 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.161928    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5hsd\" (UniqueName: \"kubernetes.io/projected/56e18a61-b310-4d01-a7b4-ff1b09e11894-kube-api-access-h5hsd\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.162074 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.162003    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-service-ca\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.162597 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.162568    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-oauth-serving-cert\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.162713 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.162618    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-console-config\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.162713 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.162682    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-service-ca\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.162860 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.162843    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-trusted-ca-bundle\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.164284 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.164258    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56e18a61-b310-4d01-a7b4-ff1b09e11894-console-serving-cert\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.164392 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.164340    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56e18a61-b310-4d01-a7b4-ff1b09e11894-console-oauth-config\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.169836 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.169812    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5hsd\" (UniqueName: \"kubernetes.io/projected/56e18a61-b310-4d01-a7b4-ff1b09e11894-kube-api-access-h5hsd\") pod \"console-5b4bcf79dc-4k8kt\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") " pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.176874 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.176845    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:16:53.358599 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.358562    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:16:53.640116 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.639984    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b4bcf79dc-4k8kt"]
Sep 10 17:16:53.642848 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:16:53.642814    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56e18a61_b310_4d01_a7b4_ff1b09e11894.slice/crio-59dbd72887df8e440149d2939122f99e7c8079cbca220ce1bf3c0046123be129 WatchSource:0}: Error finding container 59dbd72887df8e440149d2939122f99e7c8079cbca220ce1bf3c0046123be129: Status 404 returned error can't find the container with id 59dbd72887df8e440149d2939122f99e7c8079cbca220ce1bf3c0046123be129
Sep 10 17:16:53.884452 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.884395    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4bcf79dc-4k8kt" event={"ID":"56e18a61-b310-4d01-a7b4-ff1b09e11894","Type":"ContainerStarted","Data":"7b3144c0a8fb5fa2a2f2fef88eb919d6be54ef509d8655911f0f60d265abc9eb"}
Sep 10 17:16:53.884452 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:16:53.884456    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4bcf79dc-4k8kt" event={"ID":"56e18a61-b310-4d01-a7b4-ff1b09e11894","Type":"ContainerStarted","Data":"59dbd72887df8e440149d2939122f99e7c8079cbca220ce1bf3c0046123be129"}
Sep 10 17:17:03.359079 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:03.359021    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:17:03.359079 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:03.359101    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:17:03.364119 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:03.364095    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:17:03.393020 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:03.392961    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b4bcf79dc-4k8kt" podStartSLOduration=10.392943242 podStartE2EDuration="10.392943242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:16:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:16:53.909702588 +0000 UTC m=+217.590875531" watchObservedRunningTime="2026-09-10 17:17:03.392943242 +0000 UTC m=+227.074116184"
Sep 10 17:17:03.928851 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:03.928818    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:17:04.037574 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:04.037537    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78fc566c97-xwzzz"]
Sep 10 17:17:04.289397 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:04.289362    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:17:04.386579 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:04.386538    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:17:27.235121 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:27.235081    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:17:27.405739 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:27.405706    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:17:29.058149 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.058045    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78fc566c97-xwzzz" podUID="1cab00f2-5dfa-43d9-b45f-96d2f79d1811" containerName="console" containerID="cri-o://5246a4fda22c85fc184898b5492292b05eeffcabed45d67e5899d5adc1c01d0d" gracePeriod=15
Sep 10 17:17:29.298141 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.298099    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:17:29.483688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.483578    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-console-serving-cert\") pod \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") "
Sep 10 17:17:29.483688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.483626    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-trusted-ca-bundle\") pod \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") "
Sep 10 17:17:29.483688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.483654    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tq8wl\" (UniqueName: \"kubernetes.io/projected/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-kube-api-access-tq8wl\") pod \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") "
Sep 10 17:17:29.483972 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.483719    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-console-config\") pod \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") "
Sep 10 17:17:29.483972 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.483755    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-oauth-serving-cert\") pod \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") "
Sep 10 17:17:29.483972 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.483793    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-console-oauth-config\") pod \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") "
Sep 10 17:17:29.483972 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.483823    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-service-ca\") pod \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\" (UID: \"1cab00f2-5dfa-43d9-b45f-96d2f79d1811\") "
Sep 10 17:17:29.484185 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.484070    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1cab00f2-5dfa-43d9-b45f-96d2f79d1811" (UID: "1cab00f2-5dfa-43d9-b45f-96d2f79d1811"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:17:29.484247 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.484197    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1cab00f2-5dfa-43d9-b45f-96d2f79d1811" (UID: "1cab00f2-5dfa-43d9-b45f-96d2f79d1811"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:17:29.484301 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.484263    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-console-config" (OuterVolumeSpecName: "console-config") pod "1cab00f2-5dfa-43d9-b45f-96d2f79d1811" (UID: "1cab00f2-5dfa-43d9-b45f-96d2f79d1811"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:17:29.484351 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.484320    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-service-ca" (OuterVolumeSpecName: "service-ca") pod "1cab00f2-5dfa-43d9-b45f-96d2f79d1811" (UID: "1cab00f2-5dfa-43d9-b45f-96d2f79d1811"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:17:29.486099 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.486065    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1cab00f2-5dfa-43d9-b45f-96d2f79d1811" (UID: "1cab00f2-5dfa-43d9-b45f-96d2f79d1811"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:17:29.486474 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.486446    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-kube-api-access-tq8wl" (OuterVolumeSpecName: "kube-api-access-tq8wl") pod "1cab00f2-5dfa-43d9-b45f-96d2f79d1811" (UID: "1cab00f2-5dfa-43d9-b45f-96d2f79d1811"). InnerVolumeSpecName "kube-api-access-tq8wl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:17:29.486474 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.486443    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1cab00f2-5dfa-43d9-b45f-96d2f79d1811" (UID: "1cab00f2-5dfa-43d9-b45f-96d2f79d1811"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:17:29.584777 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.584730    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-console-serving-cert\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:17:29.584777 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.584767    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-trusted-ca-bundle\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:17:29.584777 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.584777    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tq8wl\" (UniqueName: \"kubernetes.io/projected/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-kube-api-access-tq8wl\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:17:29.584777 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.584787    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-console-config\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:17:29.585057 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.584797    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-oauth-serving-cert\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:17:29.585057 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.584806    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-console-oauth-config\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:17:29.585057 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:29.584815    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1cab00f2-5dfa-43d9-b45f-96d2f79d1811-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:17:30.034324 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:30.034288    2566 generic.go:356] "Generic (PLEG): container finished" podID="1cab00f2-5dfa-43d9-b45f-96d2f79d1811" containerID="5246a4fda22c85fc184898b5492292b05eeffcabed45d67e5899d5adc1c01d0d" exitCode=0
Sep 10 17:17:30.034516 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:30.034340    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78fc566c97-xwzzz"
Sep 10 17:17:30.034516 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:30.034365    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78fc566c97-xwzzz" event={"ID":"1cab00f2-5dfa-43d9-b45f-96d2f79d1811","Type":"ContainerDied","Data":"5246a4fda22c85fc184898b5492292b05eeffcabed45d67e5899d5adc1c01d0d"}
Sep 10 17:17:30.034516 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:30.034407    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78fc566c97-xwzzz" event={"ID":"1cab00f2-5dfa-43d9-b45f-96d2f79d1811","Type":"ContainerDied","Data":"66b3d40ff310feaa334aa20e9435877c90493f8d4195e166370b2c8416ca4c6a"}
Sep 10 17:17:30.034516 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:30.034427    2566 scope.go:117] "RemoveContainer" containerID="5246a4fda22c85fc184898b5492292b05eeffcabed45d67e5899d5adc1c01d0d"
Sep 10 17:17:30.043911 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:30.043888    2566 scope.go:117] "RemoveContainer" containerID="5246a4fda22c85fc184898b5492292b05eeffcabed45d67e5899d5adc1c01d0d"
Sep 10 17:17:30.044288 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:17:30.044269    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5246a4fda22c85fc184898b5492292b05eeffcabed45d67e5899d5adc1c01d0d\": container with ID starting with 5246a4fda22c85fc184898b5492292b05eeffcabed45d67e5899d5adc1c01d0d not found: ID does not exist" containerID="5246a4fda22c85fc184898b5492292b05eeffcabed45d67e5899d5adc1c01d0d"
Sep 10 17:17:30.044346 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:30.044299    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5246a4fda22c85fc184898b5492292b05eeffcabed45d67e5899d5adc1c01d0d"} err="failed to get container status \"5246a4fda22c85fc184898b5492292b05eeffcabed45d67e5899d5adc1c01d0d\": rpc error: code = NotFound desc = could not find container \"5246a4fda22c85fc184898b5492292b05eeffcabed45d67e5899d5adc1c01d0d\": container with ID starting with 5246a4fda22c85fc184898b5492292b05eeffcabed45d67e5899d5adc1c01d0d not found: ID does not exist"
Sep 10 17:17:30.056586 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:30.056542    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78fc566c97-xwzzz"]
Sep 10 17:17:30.060538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:30.060495    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78fc566c97-xwzzz"]
Sep 10 17:17:30.833803 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:30.833765    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cab00f2-5dfa-43d9-b45f-96d2f79d1811" path="/var/lib/kubelet/pods/1cab00f2-5dfa-43d9-b45f-96d2f79d1811/volumes"
Sep 10 17:17:59.992104 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:17:59.992055    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:18:00.100688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:00.100635    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:18:12.077703 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:12.077657    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:18:12.150569 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:12.150532    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:18:16.750496 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:16.750465    2566 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 17:18:23.358886 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:23.358829    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:18:23.503029 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:23.502988    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:18:38.290680 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.290636    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85dcfddb4d-cvqqs"]
Sep 10 17:18:38.291334 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.291013    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cab00f2-5dfa-43d9-b45f-96d2f79d1811" containerName="console"
Sep 10 17:18:38.291334 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.291027    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cab00f2-5dfa-43d9-b45f-96d2f79d1811" containerName="console"
Sep 10 17:18:38.291334 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.291086    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cab00f2-5dfa-43d9-b45f-96d2f79d1811" containerName="console"
Sep 10 17:18:38.294117 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.294097    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.307592 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.307555    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85dcfddb4d-cvqqs"]
Sep 10 17:18:38.368496 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.368455    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-service-ca\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.368678 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.368544    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-trusted-ca-bundle\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.368678 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.368570    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c64a49b1-c790-4f19-8703-21b6a7157e3d-console-serving-cert\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.368678 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.368617    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm2sl\" (UniqueName: \"kubernetes.io/projected/c64a49b1-c790-4f19-8703-21b6a7157e3d-kube-api-access-dm2sl\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.368678 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.368649    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c64a49b1-c790-4f19-8703-21b6a7157e3d-console-oauth-config\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.368815 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.368703    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-console-config\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.368815 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.368757    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-oauth-serving-cert\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.372672 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.372633    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:18:38.469551 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.469511    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-oauth-serving-cert\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.469743 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.469677    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-service-ca\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.469809 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.469766    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-trusted-ca-bundle\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.469809 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.469800    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c64a49b1-c790-4f19-8703-21b6a7157e3d-console-serving-cert\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.469917 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.469832    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm2sl\" (UniqueName: \"kubernetes.io/projected/c64a49b1-c790-4f19-8703-21b6a7157e3d-kube-api-access-dm2sl\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.469917 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.469870    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c64a49b1-c790-4f19-8703-21b6a7157e3d-console-oauth-config\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.469917 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.469896    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-console-config\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.470444 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.470412    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-oauth-serving-cert\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.470563 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.470469    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-service-ca\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.470832 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.470808    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-trusted-ca-bundle\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.470913 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.470817    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-console-config\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.472160 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.472112    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:18:38.472776 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.472756    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c64a49b1-c790-4f19-8703-21b6a7157e3d-console-oauth-config\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.472776 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.472769    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c64a49b1-c790-4f19-8703-21b6a7157e3d-console-serving-cert\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.478383 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.478352    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm2sl\" (UniqueName: \"kubernetes.io/projected/c64a49b1-c790-4f19-8703-21b6a7157e3d-kube-api-access-dm2sl\") pod \"console-85dcfddb4d-cvqqs\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") " pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.603820 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.603749    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:38.880731 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:38.880704    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85dcfddb4d-cvqqs"]
Sep 10 17:18:38.884441 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:18:38.884406    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc64a49b1_c790_4f19_8703_21b6a7157e3d.slice/crio-11254c5cfb0cbadee084f989cc74c577e3879dcff0b40073c4afe45252cf7a3b WatchSource:0}: Error finding container 11254c5cfb0cbadee084f989cc74c577e3879dcff0b40073c4afe45252cf7a3b: Status 404 returned error can't find the container with id 11254c5cfb0cbadee084f989cc74c577e3879dcff0b40073c4afe45252cf7a3b
Sep 10 17:18:39.325976 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:39.325939    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85dcfddb4d-cvqqs" event={"ID":"c64a49b1-c790-4f19-8703-21b6a7157e3d","Type":"ContainerStarted","Data":"a2bf800f98374f07bd1535c15fb2df30301cfcc562012cb881c6f18f93be0d6f"}
Sep 10 17:18:39.326391 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:39.326002    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85dcfddb4d-cvqqs" event={"ID":"c64a49b1-c790-4f19-8703-21b6a7157e3d","Type":"ContainerStarted","Data":"11254c5cfb0cbadee084f989cc74c577e3879dcff0b40073c4afe45252cf7a3b"}
Sep 10 17:18:39.349302 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:39.349243    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85dcfddb4d-cvqqs" podStartSLOduration=1.3492252470000001 podStartE2EDuration="1.349225247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:18:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:18:39.347199146 +0000 UTC m=+323.028372091" watchObservedRunningTime="2026-09-10 17:18:39.349225247 +0000 UTC m=+323.030398189"
Sep 10 17:18:48.603928 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:48.603883    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:48.604435 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:48.603974    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:48.608926 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:48.608900    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:49.377105 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:49.377073    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:18:49.433818 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:49.433782    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b4bcf79dc-4k8kt"]
Sep 10 17:18:49.591012 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:49.590966    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:18:49.689799 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:18:49.689716    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:04.339347 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:04.339314    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:05.573939 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:05.573905    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:05.574542 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:05.574521    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh"]
Sep 10 17:19:05.578217 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:05.578199    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh"
Sep 10 17:19:05.580822 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:05.580800    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 17:19:05.581722 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:05.581703    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 17:19:05.581792 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:05.581708    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9gr7t\""
Sep 10 17:19:05.591173 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:05.591154    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh"]
Sep 10 17:19:05.704457 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:05.704428    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e39277d-110c-4d7a-b95f-ce33d65b7de0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh\" (UID: \"2e39277d-110c-4d7a-b95f-ce33d65b7de0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh"
Sep 10 17:19:05.704603 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:05.704499    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e39277d-110c-4d7a-b95f-ce33d65b7de0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh\" (UID: \"2e39277d-110c-4d7a-b95f-ce33d65b7de0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh"
Sep 10 17:19:05.704603 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:05.704527    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r72hj\" (UniqueName: \"kubernetes.io/projected/2e39277d-110c-4d7a-b95f-ce33d65b7de0-kube-api-access-r72hj\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh\" (UID: \"2e39277d-110c-4d7a-b95f-ce33d65b7de0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh"
Sep 10 17:19:05.805197 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:05.805169    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e39277d-110c-4d7a-b95f-ce33d65b7de0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh\" (UID: \"2e39277d-110c-4d7a-b95f-ce33d65b7de0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh"
Sep 10 17:19:05.805298 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:05.805233    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e39277d-110c-4d7a-b95f-ce33d65b7de0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh\" (UID: \"2e39277d-110c-4d7a-b95f-ce33d65b7de0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh"
Sep 10 17:19:05.805298 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:05.805252    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r72hj\" (UniqueName: \"kubernetes.io/projected/2e39277d-110c-4d7a-b95f-ce33d65b7de0-kube-api-access-r72hj\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh\" (UID: \"2e39277d-110c-4d7a-b95f-ce33d65b7de0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh"
Sep 10 17:19:05.805529 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:05.805510    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e39277d-110c-4d7a-b95f-ce33d65b7de0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh\" (UID: \"2e39277d-110c-4d7a-b95f-ce33d65b7de0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh"
Sep 10 17:19:05.805630 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:05.805612    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e39277d-110c-4d7a-b95f-ce33d65b7de0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh\" (UID: \"2e39277d-110c-4d7a-b95f-ce33d65b7de0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh"
Sep 10 17:19:05.814345 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:05.814317    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r72hj\" (UniqueName: \"kubernetes.io/projected/2e39277d-110c-4d7a-b95f-ce33d65b7de0-kube-api-access-r72hj\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh\" (UID: \"2e39277d-110c-4d7a-b95f-ce33d65b7de0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh"
Sep 10 17:19:05.887234 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:05.887179    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh"
Sep 10 17:19:06.158640 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:06.158610    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh"]
Sep 10 17:19:06.161109 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:19:06.161084    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e39277d_110c_4d7a_b95f_ce33d65b7de0.slice/crio-a42fe65149a753e05abff9f606438e0fdae65efef2353251fa118c097f7c90fc WatchSource:0}: Error finding container a42fe65149a753e05abff9f606438e0fdae65efef2353251fa118c097f7c90fc: Status 404 returned error can't find the container with id a42fe65149a753e05abff9f606438e0fdae65efef2353251fa118c097f7c90fc
Sep 10 17:19:06.163120 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:06.163104    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 17:19:06.440800 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:06.440718    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh" event={"ID":"2e39277d-110c-4d7a-b95f-ce33d65b7de0","Type":"ContainerStarted","Data":"a42fe65149a753e05abff9f606438e0fdae65efef2353251fa118c097f7c90fc"}
Sep 10 17:19:13.474758 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:13.474725    2566 generic.go:356] "Generic (PLEG): container finished" podID="2e39277d-110c-4d7a-b95f-ce33d65b7de0" containerID="2971cf0dd752f328005a731fa82049e9aa8f6a81312a1661816ddbc7c1b2322c" exitCode=0
Sep 10 17:19:13.475236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:13.474810    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh" event={"ID":"2e39277d-110c-4d7a-b95f-ce33d65b7de0","Type":"ContainerDied","Data":"2971cf0dd752f328005a731fa82049e9aa8f6a81312a1661816ddbc7c1b2322c"}
Sep 10 17:19:14.454517 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.454468    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b4bcf79dc-4k8kt" podUID="56e18a61-b310-4d01-a7b4-ff1b09e11894" containerName="console" containerID="cri-o://7b3144c0a8fb5fa2a2f2fef88eb919d6be54ef509d8655911f0f60d265abc9eb" gracePeriod=15
Sep 10 17:19:14.712099 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.712041    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:19:14.782377 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.782344    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5hsd\" (UniqueName: \"kubernetes.io/projected/56e18a61-b310-4d01-a7b4-ff1b09e11894-kube-api-access-h5hsd\") pod \"56e18a61-b310-4d01-a7b4-ff1b09e11894\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") "
Sep 10 17:19:14.782541 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.782393    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-service-ca\") pod \"56e18a61-b310-4d01-a7b4-ff1b09e11894\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") "
Sep 10 17:19:14.782541 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.782439    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56e18a61-b310-4d01-a7b4-ff1b09e11894-console-oauth-config\") pod \"56e18a61-b310-4d01-a7b4-ff1b09e11894\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") "
Sep 10 17:19:14.782661 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.782579    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-trusted-ca-bundle\") pod \"56e18a61-b310-4d01-a7b4-ff1b09e11894\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") "
Sep 10 17:19:14.782714 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.782691    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-oauth-serving-cert\") pod \"56e18a61-b310-4d01-a7b4-ff1b09e11894\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") "
Sep 10 17:19:14.782764 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.782717    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56e18a61-b310-4d01-a7b4-ff1b09e11894-console-serving-cert\") pod \"56e18a61-b310-4d01-a7b4-ff1b09e11894\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") "
Sep 10 17:19:14.782764 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.782746    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-console-config\") pod \"56e18a61-b310-4d01-a7b4-ff1b09e11894\" (UID: \"56e18a61-b310-4d01-a7b4-ff1b09e11894\") "
Sep 10 17:19:14.782871 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.782856    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-service-ca" (OuterVolumeSpecName: "service-ca") pod "56e18a61-b310-4d01-a7b4-ff1b09e11894" (UID: "56e18a61-b310-4d01-a7b4-ff1b09e11894"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:19:14.783104 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.783081    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:19:14.783215 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.783076    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "56e18a61-b310-4d01-a7b4-ff1b09e11894" (UID: "56e18a61-b310-4d01-a7b4-ff1b09e11894"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:19:14.783215 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.783112    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "56e18a61-b310-4d01-a7b4-ff1b09e11894" (UID: "56e18a61-b310-4d01-a7b4-ff1b09e11894"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:19:14.783215 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.783178    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-console-config" (OuterVolumeSpecName: "console-config") pod "56e18a61-b310-4d01-a7b4-ff1b09e11894" (UID: "56e18a61-b310-4d01-a7b4-ff1b09e11894"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:19:14.784958 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.784930    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56e18a61-b310-4d01-a7b4-ff1b09e11894-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "56e18a61-b310-4d01-a7b4-ff1b09e11894" (UID: "56e18a61-b310-4d01-a7b4-ff1b09e11894"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:19:14.785081 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.784955    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56e18a61-b310-4d01-a7b4-ff1b09e11894-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "56e18a61-b310-4d01-a7b4-ff1b09e11894" (UID: "56e18a61-b310-4d01-a7b4-ff1b09e11894"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:19:14.785233 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.785204    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56e18a61-b310-4d01-a7b4-ff1b09e11894-kube-api-access-h5hsd" (OuterVolumeSpecName: "kube-api-access-h5hsd") pod "56e18a61-b310-4d01-a7b4-ff1b09e11894" (UID: "56e18a61-b310-4d01-a7b4-ff1b09e11894"). InnerVolumeSpecName "kube-api-access-h5hsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:19:14.884842 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.884599    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56e18a61-b310-4d01-a7b4-ff1b09e11894-console-oauth-config\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:19:14.884842 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.884628    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-trusted-ca-bundle\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:19:14.884842 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.884653    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-oauth-serving-cert\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:19:14.884842 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.884675    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56e18a61-b310-4d01-a7b4-ff1b09e11894-console-serving-cert\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:19:14.884842 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.884691    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56e18a61-b310-4d01-a7b4-ff1b09e11894-console-config\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:19:14.884842 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:14.884705    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5hsd\" (UniqueName: \"kubernetes.io/projected/56e18a61-b310-4d01-a7b4-ff1b09e11894-kube-api-access-h5hsd\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:19:15.485698 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:15.485656    2566 generic.go:356] "Generic (PLEG): container finished" podID="56e18a61-b310-4d01-a7b4-ff1b09e11894" containerID="7b3144c0a8fb5fa2a2f2fef88eb919d6be54ef509d8655911f0f60d265abc9eb" exitCode=0
Sep 10 17:19:15.485867 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:15.485721    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4bcf79dc-4k8kt" event={"ID":"56e18a61-b310-4d01-a7b4-ff1b09e11894","Type":"ContainerDied","Data":"7b3144c0a8fb5fa2a2f2fef88eb919d6be54ef509d8655911f0f60d265abc9eb"}
Sep 10 17:19:15.485867 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:15.485727    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4bcf79dc-4k8kt"
Sep 10 17:19:15.485867 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:15.485804    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4bcf79dc-4k8kt" event={"ID":"56e18a61-b310-4d01-a7b4-ff1b09e11894","Type":"ContainerDied","Data":"59dbd72887df8e440149d2939122f99e7c8079cbca220ce1bf3c0046123be129"}
Sep 10 17:19:15.485867 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:15.485844    2566 scope.go:117] "RemoveContainer" containerID="7b3144c0a8fb5fa2a2f2fef88eb919d6be54ef509d8655911f0f60d265abc9eb"
Sep 10 17:19:15.515550 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:15.515518    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b4bcf79dc-4k8kt"]
Sep 10 17:19:15.519023 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:15.518997    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b4bcf79dc-4k8kt"]
Sep 10 17:19:15.610110 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:15.609960    2566 scope.go:117] "RemoveContainer" containerID="7b3144c0a8fb5fa2a2f2fef88eb919d6be54ef509d8655911f0f60d265abc9eb"
Sep 10 17:19:15.610338 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:19:15.610316    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b3144c0a8fb5fa2a2f2fef88eb919d6be54ef509d8655911f0f60d265abc9eb\": container with ID starting with 7b3144c0a8fb5fa2a2f2fef88eb919d6be54ef509d8655911f0f60d265abc9eb not found: ID does not exist" containerID="7b3144c0a8fb5fa2a2f2fef88eb919d6be54ef509d8655911f0f60d265abc9eb"
Sep 10 17:19:15.610386 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:15.610348    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b3144c0a8fb5fa2a2f2fef88eb919d6be54ef509d8655911f0f60d265abc9eb"} err="failed to get container status \"7b3144c0a8fb5fa2a2f2fef88eb919d6be54ef509d8655911f0f60d265abc9eb\": rpc error: code = NotFound desc = could not find container \"7b3144c0a8fb5fa2a2f2fef88eb919d6be54ef509d8655911f0f60d265abc9eb\": container with ID starting with 7b3144c0a8fb5fa2a2f2fef88eb919d6be54ef509d8655911f0f60d265abc9eb not found: ID does not exist"
Sep 10 17:19:16.491374 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:16.491340    2566 generic.go:356] "Generic (PLEG): container finished" podID="2e39277d-110c-4d7a-b95f-ce33d65b7de0" containerID="fdbc5bcdc34d44631dbc70c9a06eb26dd07e811e3f9f75c011c44da11c68699c" exitCode=0
Sep 10 17:19:16.491784 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:16.491432    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh" event={"ID":"2e39277d-110c-4d7a-b95f-ce33d65b7de0","Type":"ContainerDied","Data":"fdbc5bcdc34d44631dbc70c9a06eb26dd07e811e3f9f75c011c44da11c68699c"}
Sep 10 17:19:16.836311 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:16.836278    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56e18a61-b310-4d01-a7b4-ff1b09e11894" path="/var/lib/kubelet/pods/56e18a61-b310-4d01-a7b4-ff1b09e11894/volumes"
Sep 10 17:19:23.526509 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:23.526473    2566 generic.go:356] "Generic (PLEG): container finished" podID="2e39277d-110c-4d7a-b95f-ce33d65b7de0" containerID="c92d488f3a5359ae9eb43937face9e1e29b4edfdf5f8c6f479961c9908f9cf22" exitCode=0
Sep 10 17:19:23.526929 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:23.526527    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh" event={"ID":"2e39277d-110c-4d7a-b95f-ce33d65b7de0","Type":"ContainerDied","Data":"c92d488f3a5359ae9eb43937face9e1e29b4edfdf5f8c6f479961c9908f9cf22"}
Sep 10 17:19:24.701343 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:24.701317    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh"
Sep 10 17:19:24.770560 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:24.770534    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e39277d-110c-4d7a-b95f-ce33d65b7de0-bundle\") pod \"2e39277d-110c-4d7a-b95f-ce33d65b7de0\" (UID: \"2e39277d-110c-4d7a-b95f-ce33d65b7de0\") "
Sep 10 17:19:24.770684 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:24.770575    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r72hj\" (UniqueName: \"kubernetes.io/projected/2e39277d-110c-4d7a-b95f-ce33d65b7de0-kube-api-access-r72hj\") pod \"2e39277d-110c-4d7a-b95f-ce33d65b7de0\" (UID: \"2e39277d-110c-4d7a-b95f-ce33d65b7de0\") "
Sep 10 17:19:24.770684 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:24.770620    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e39277d-110c-4d7a-b95f-ce33d65b7de0-util\") pod \"2e39277d-110c-4d7a-b95f-ce33d65b7de0\" (UID: \"2e39277d-110c-4d7a-b95f-ce33d65b7de0\") "
Sep 10 17:19:24.771234 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:24.771200    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e39277d-110c-4d7a-b95f-ce33d65b7de0-bundle" (OuterVolumeSpecName: "bundle") pod "2e39277d-110c-4d7a-b95f-ce33d65b7de0" (UID: "2e39277d-110c-4d7a-b95f-ce33d65b7de0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:19:24.772719 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:24.772694    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e39277d-110c-4d7a-b95f-ce33d65b7de0-kube-api-access-r72hj" (OuterVolumeSpecName: "kube-api-access-r72hj") pod "2e39277d-110c-4d7a-b95f-ce33d65b7de0" (UID: "2e39277d-110c-4d7a-b95f-ce33d65b7de0"). InnerVolumeSpecName "kube-api-access-r72hj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:19:24.777132 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:24.777101    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e39277d-110c-4d7a-b95f-ce33d65b7de0-util" (OuterVolumeSpecName: "util") pod "2e39277d-110c-4d7a-b95f-ce33d65b7de0" (UID: "2e39277d-110c-4d7a-b95f-ce33d65b7de0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:19:24.871896 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:24.871870    2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e39277d-110c-4d7a-b95f-ce33d65b7de0-util\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:19:24.871896 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:24.871894    2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e39277d-110c-4d7a-b95f-ce33d65b7de0-bundle\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:19:24.872028 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:24.871908    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r72hj\" (UniqueName: \"kubernetes.io/projected/2e39277d-110c-4d7a-b95f-ce33d65b7de0-kube-api-access-r72hj\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:19:25.535823 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:25.535794    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh" event={"ID":"2e39277d-110c-4d7a-b95f-ce33d65b7de0","Type":"ContainerDied","Data":"a42fe65149a753e05abff9f606438e0fdae65efef2353251fa118c097f7c90fc"}
Sep 10 17:19:25.535823 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:25.535808    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zftnh"
Sep 10 17:19:25.536039 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:25.535837    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a42fe65149a753e05abff9f606438e0fdae65efef2353251fa118c097f7c90fc"
Sep 10 17:19:26.652248 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:26.652206    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:27.544282 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:27.544246    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:28.871355 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:28.871322    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fgv9f"]
Sep 10 17:19:28.871810 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:28.871651    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e39277d-110c-4d7a-b95f-ce33d65b7de0" containerName="util"
Sep 10 17:19:28.871810 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:28.871661    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e39277d-110c-4d7a-b95f-ce33d65b7de0" containerName="util"
Sep 10 17:19:28.871810 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:28.871674    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e39277d-110c-4d7a-b95f-ce33d65b7de0" containerName="pull"
Sep 10 17:19:28.871810 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:28.871679    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e39277d-110c-4d7a-b95f-ce33d65b7de0" containerName="pull"
Sep 10 17:19:28.871810 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:28.871692    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e39277d-110c-4d7a-b95f-ce33d65b7de0" containerName="extract"
Sep 10 17:19:28.871810 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:28.871697    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e39277d-110c-4d7a-b95f-ce33d65b7de0" containerName="extract"
Sep 10 17:19:28.871810 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:28.871707    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56e18a61-b310-4d01-a7b4-ff1b09e11894" containerName="console"
Sep 10 17:19:28.871810 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:28.871712    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="56e18a61-b310-4d01-a7b4-ff1b09e11894" containerName="console"
Sep 10 17:19:28.871810 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:28.871764    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e39277d-110c-4d7a-b95f-ce33d65b7de0" containerName="extract"
Sep 10 17:19:28.871810 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:28.871770    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="56e18a61-b310-4d01-a7b4-ff1b09e11894" containerName="console"
Sep 10 17:19:28.875005 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:28.874990    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fgv9f"
Sep 10 17:19:28.878010 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:28.877989    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 10 17:19:28.878163 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:28.878086    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-8rsxk\""
Sep 10 17:19:28.878299 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:28.878284    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 10 17:19:28.905365 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:28.905344    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fgv9f"]
Sep 10 17:19:29.002320 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:29.002289    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88c7b58d-c592-4e25-9c39-ab29bd606fa1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fgv9f\" (UID: \"88c7b58d-c592-4e25-9c39-ab29bd606fa1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fgv9f"
Sep 10 17:19:29.002439 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:29.002375    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76228\" (UniqueName: \"kubernetes.io/projected/88c7b58d-c592-4e25-9c39-ab29bd606fa1-kube-api-access-76228\") pod \"cert-manager-operator-controller-manager-55c444bf86-fgv9f\" (UID: \"88c7b58d-c592-4e25-9c39-ab29bd606fa1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fgv9f"
Sep 10 17:19:29.102915 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:29.102888    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76228\" (UniqueName: \"kubernetes.io/projected/88c7b58d-c592-4e25-9c39-ab29bd606fa1-kube-api-access-76228\") pod \"cert-manager-operator-controller-manager-55c444bf86-fgv9f\" (UID: \"88c7b58d-c592-4e25-9c39-ab29bd606fa1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fgv9f"
Sep 10 17:19:29.103030 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:29.102927    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88c7b58d-c592-4e25-9c39-ab29bd606fa1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fgv9f\" (UID: \"88c7b58d-c592-4e25-9c39-ab29bd606fa1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fgv9f"
Sep 10 17:19:29.103279 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:29.103264    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88c7b58d-c592-4e25-9c39-ab29bd606fa1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fgv9f\" (UID: \"88c7b58d-c592-4e25-9c39-ab29bd606fa1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fgv9f"
Sep 10 17:19:29.111623 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:29.111599    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76228\" (UniqueName: \"kubernetes.io/projected/88c7b58d-c592-4e25-9c39-ab29bd606fa1-kube-api-access-76228\") pod \"cert-manager-operator-controller-manager-55c444bf86-fgv9f\" (UID: \"88c7b58d-c592-4e25-9c39-ab29bd606fa1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fgv9f"
Sep 10 17:19:29.184297 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:29.184245    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fgv9f"
Sep 10 17:19:29.470881 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:29.470858    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fgv9f"]
Sep 10 17:19:29.473240 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:19:29.473208    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88c7b58d_c592_4e25_9c39_ab29bd606fa1.slice/crio-81d2b5fc0d67bd839bc2faf30dd3854fc102df8cad76c51373e3d86fbb0c02df WatchSource:0}: Error finding container 81d2b5fc0d67bd839bc2faf30dd3854fc102df8cad76c51373e3d86fbb0c02df: Status 404 returned error can't find the container with id 81d2b5fc0d67bd839bc2faf30dd3854fc102df8cad76c51373e3d86fbb0c02df
Sep 10 17:19:29.554962 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:29.554933    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fgv9f" event={"ID":"88c7b58d-c592-4e25-9c39-ab29bd606fa1","Type":"ContainerStarted","Data":"81d2b5fc0d67bd839bc2faf30dd3854fc102df8cad76c51373e3d86fbb0c02df"}
Sep 10 17:19:30.669059 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:30.669001    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:30.767597 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:30.767562    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:31.566132 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:31.566078    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fgv9f" event={"ID":"88c7b58d-c592-4e25-9c39-ab29bd606fa1","Type":"ContainerStarted","Data":"deb5215eb12e675d26d91c6e1434e852e9984cbea1d69102010523285462c2be"}
Sep 10 17:19:31.606086 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:31.606038    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fgv9f" podStartSLOduration=1.717090426 podStartE2EDuration="3.606020199s" totalImagesPullingTime="1.888929773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:19:31.605925219 +0000 UTC m=+375.287098182" watchObservedRunningTime="2026-09-10 17:19:31.606020199 +0000 UTC m=+375.287193142"
Sep 10 17:19:32.145475 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:32.145438    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:32.351777 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:32.351734    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:32.804175 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:32.804117    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx"]
Sep 10 17:19:32.807864 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:32.807844    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx"
Sep 10 17:19:32.826120 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:32.826102    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 17:19:32.826235 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:32.826214    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9gr7t\""
Sep 10 17:19:32.827061 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:32.827047    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 17:19:32.837959 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:32.837931    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx"]
Sep 10 17:19:32.936456 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:32.936428    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsdwm\" (UniqueName: \"kubernetes.io/projected/e3f3e38b-6d52-4beb-886f-ad3ddf93008f-kube-api-access-dsdwm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx\" (UID: \"e3f3e38b-6d52-4beb-886f-ad3ddf93008f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx"
Sep 10 17:19:32.936636 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:32.936462    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3f3e38b-6d52-4beb-886f-ad3ddf93008f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx\" (UID: \"e3f3e38b-6d52-4beb-886f-ad3ddf93008f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx"
Sep 10 17:19:32.936636 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:32.936499    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3f3e38b-6d52-4beb-886f-ad3ddf93008f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx\" (UID: \"e3f3e38b-6d52-4beb-886f-ad3ddf93008f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx"
Sep 10 17:19:32.992907 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:32.992870    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:33.037399 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.037366    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsdwm\" (UniqueName: \"kubernetes.io/projected/e3f3e38b-6d52-4beb-886f-ad3ddf93008f-kube-api-access-dsdwm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx\" (UID: \"e3f3e38b-6d52-4beb-886f-ad3ddf93008f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx"
Sep 10 17:19:33.037529 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.037404    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3f3e38b-6d52-4beb-886f-ad3ddf93008f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx\" (UID: \"e3f3e38b-6d52-4beb-886f-ad3ddf93008f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx"
Sep 10 17:19:33.037529 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.037443    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3f3e38b-6d52-4beb-886f-ad3ddf93008f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx\" (UID: \"e3f3e38b-6d52-4beb-886f-ad3ddf93008f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx"
Sep 10 17:19:33.037759 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.037744    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3f3e38b-6d52-4beb-886f-ad3ddf93008f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx\" (UID: \"e3f3e38b-6d52-4beb-886f-ad3ddf93008f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx"
Sep 10 17:19:33.037800 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.037778    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3f3e38b-6d52-4beb-886f-ad3ddf93008f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx\" (UID: \"e3f3e38b-6d52-4beb-886f-ad3ddf93008f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx"
Sep 10 17:19:33.068248 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.068186    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsdwm\" (UniqueName: \"kubernetes.io/projected/e3f3e38b-6d52-4beb-886f-ad3ddf93008f-kube-api-access-dsdwm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx\" (UID: \"e3f3e38b-6d52-4beb-886f-ad3ddf93008f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx"
Sep 10 17:19:33.117013 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.116981    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx"
Sep 10 17:19:33.129643 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.129611    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:33.222619 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.222591    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-55gq5"]
Sep 10 17:19:33.226853 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.226837    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-55gq5"
Sep 10 17:19:33.229331 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.229309    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 17:19:33.229696 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.229682    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 17:19:33.255890 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.255868    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-55gq5"]
Sep 10 17:19:33.340829 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.340759    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vw5w\" (UniqueName: \"kubernetes.io/projected/c41dd511-8076-4d7e-ace0-7c81f268e0a5-kube-api-access-5vw5w\") pod \"cert-manager-webhook-6486ffd997-55gq5\" (UID: \"c41dd511-8076-4d7e-ace0-7c81f268e0a5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-55gq5"
Sep 10 17:19:33.340829 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.340803    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c41dd511-8076-4d7e-ace0-7c81f268e0a5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-55gq5\" (UID: \"c41dd511-8076-4d7e-ace0-7c81f268e0a5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-55gq5"
Sep 10 17:19:33.406963 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:19:33.406931    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3f3e38b_6d52_4beb_886f_ad3ddf93008f.slice/crio-bcbb6801e0b9c78822f4e690629305e0d11d5d0e5d4b858b92dcdf0b6ea3b2fd WatchSource:0}: Error finding container bcbb6801e0b9c78822f4e690629305e0d11d5d0e5d4b858b92dcdf0b6ea3b2fd: Status 404 returned error can't find the container with id bcbb6801e0b9c78822f4e690629305e0d11d5d0e5d4b858b92dcdf0b6ea3b2fd
Sep 10 17:19:33.407854 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.407830    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx"]
Sep 10 17:19:33.441440 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.441416    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c41dd511-8076-4d7e-ace0-7c81f268e0a5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-55gq5\" (UID: \"c41dd511-8076-4d7e-ace0-7c81f268e0a5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-55gq5"
Sep 10 17:19:33.441523 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.441483    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vw5w\" (UniqueName: \"kubernetes.io/projected/c41dd511-8076-4d7e-ace0-7c81f268e0a5-kube-api-access-5vw5w\") pod \"cert-manager-webhook-6486ffd997-55gq5\" (UID: \"c41dd511-8076-4d7e-ace0-7c81f268e0a5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-55gq5"
Sep 10 17:19:33.451989 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.451962    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vw5w\" (UniqueName: \"kubernetes.io/projected/c41dd511-8076-4d7e-ace0-7c81f268e0a5-kube-api-access-5vw5w\") pod \"cert-manager-webhook-6486ffd997-55gq5\" (UID: \"c41dd511-8076-4d7e-ace0-7c81f268e0a5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-55gq5"
Sep 10 17:19:33.452875 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.452859    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c41dd511-8076-4d7e-ace0-7c81f268e0a5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-55gq5\" (UID: \"c41dd511-8076-4d7e-ace0-7c81f268e0a5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-55gq5"
Sep 10 17:19:33.536208 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.536182    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-55gq5"
Sep 10 17:19:33.577311 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.577280    2566 generic.go:356] "Generic (PLEG): container finished" podID="e3f3e38b-6d52-4beb-886f-ad3ddf93008f" containerID="ad8a864db8941224512836bd24a66742500e20e16a8e3b6d80dbefa7b121d6dd" exitCode=0
Sep 10 17:19:33.577483 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.577367    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx" event={"ID":"e3f3e38b-6d52-4beb-886f-ad3ddf93008f","Type":"ContainerDied","Data":"ad8a864db8941224512836bd24a66742500e20e16a8e3b6d80dbefa7b121d6dd"}
Sep 10 17:19:33.577483 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.577432    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx" event={"ID":"e3f3e38b-6d52-4beb-886f-ad3ddf93008f","Type":"ContainerStarted","Data":"bcbb6801e0b9c78822f4e690629305e0d11d5d0e5d4b858b92dcdf0b6ea3b2fd"}
Sep 10 17:19:33.811850 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:33.811827    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-55gq5"]
Sep 10 17:19:33.814270 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:19:33.814245    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc41dd511_8076_4d7e_ace0_7c81f268e0a5.slice/crio-8c412bfb954963f1cad4f821c8228e156638119b3ccf6445904beae4c674eaf7 WatchSource:0}: Error finding container 8c412bfb954963f1cad4f821c8228e156638119b3ccf6445904beae4c674eaf7: Status 404 returned error can't find the container with id 8c412bfb954963f1cad4f821c8228e156638119b3ccf6445904beae4c674eaf7
Sep 10 17:19:34.586703 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:34.586664    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-55gq5" event={"ID":"c41dd511-8076-4d7e-ace0-7c81f268e0a5","Type":"ContainerStarted","Data":"8c412bfb954963f1cad4f821c8228e156638119b3ccf6445904beae4c674eaf7"}
Sep 10 17:19:34.701631 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:34.701585    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:35.524005 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:35.523919    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:36.599381 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:36.599345    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx" event={"ID":"e3f3e38b-6d52-4beb-886f-ad3ddf93008f","Type":"ContainerStarted","Data":"7c809f57901eda1db0ff9b28ad630a42f18056d24af7a6321d02e86ea8edd43c"}
Sep 10 17:19:37.484790 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:37.484754    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:37.580448 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:37.580420    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:37.605843 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:37.605815    2566 generic.go:356] "Generic (PLEG): container finished" podID="e3f3e38b-6d52-4beb-886f-ad3ddf93008f" containerID="7c809f57901eda1db0ff9b28ad630a42f18056d24af7a6321d02e86ea8edd43c" exitCode=0
Sep 10 17:19:37.606267 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:37.605887    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx" event={"ID":"e3f3e38b-6d52-4beb-886f-ad3ddf93008f","Type":"ContainerDied","Data":"7c809f57901eda1db0ff9b28ad630a42f18056d24af7a6321d02e86ea8edd43c"}
Sep 10 17:19:37.607642 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:37.607624    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-55gq5" event={"ID":"c41dd511-8076-4d7e-ace0-7c81f268e0a5","Type":"ContainerStarted","Data":"db44a0705d7f9847e4a5c7fb48c15114b5e2cb9ed8a48588b03ed6d226f2ae58"}
Sep 10 17:19:37.607751 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:37.607673    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-55gq5"
Sep 10 17:19:37.661368 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:37.661325    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-55gq5" podStartSLOduration=1.5903821360000001 podStartE2EDuration="4.661309748s" totalImagesPullingTime="3.070927612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:19:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:19:37.659932957 +0000 UTC m=+381.341105901" watchObservedRunningTime="2026-09-10 17:19:37.661309748 +0000 UTC m=+381.342482691"
Sep 10 17:19:38.614526 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:38.614491    2566 generic.go:356] "Generic (PLEG): container finished" podID="e3f3e38b-6d52-4beb-886f-ad3ddf93008f" containerID="5481301da43dec40521fb458d2d3f73f348ef8a7a1b7d16fd30b0c52895f4300" exitCode=0
Sep 10 17:19:38.614894 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:38.614556    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx" event={"ID":"e3f3e38b-6d52-4beb-886f-ad3ddf93008f","Type":"ContainerDied","Data":"5481301da43dec40521fb458d2d3f73f348ef8a7a1b7d16fd30b0c52895f4300"}
Sep 10 17:19:39.672400 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.672369    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kvn7s"]
Sep 10 17:19:39.675921 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.675904    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kvn7s"
Sep 10 17:19:39.678490 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.678469    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-br8jz\""
Sep 10 17:19:39.689447 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.689386    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kvn7s"]
Sep 10 17:19:39.745218 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.745199    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx"
Sep 10 17:19:39.797261 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.797233    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8e21d82f-ef27-4e57-8556-3aba57c3e09d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kvn7s\" (UID: \"8e21d82f-ef27-4e57-8556-3aba57c3e09d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kvn7s"
Sep 10 17:19:39.797379 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.797306    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr8fm\" (UniqueName: \"kubernetes.io/projected/8e21d82f-ef27-4e57-8556-3aba57c3e09d-kube-api-access-cr8fm\") pod \"cert-manager-cainjector-559cdd8c87-kvn7s\" (UID: \"8e21d82f-ef27-4e57-8556-3aba57c3e09d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kvn7s"
Sep 10 17:19:39.898375 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.898315    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3f3e38b-6d52-4beb-886f-ad3ddf93008f-bundle\") pod \"e3f3e38b-6d52-4beb-886f-ad3ddf93008f\" (UID: \"e3f3e38b-6d52-4beb-886f-ad3ddf93008f\") "
Sep 10 17:19:39.898491 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.898387    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsdwm\" (UniqueName: \"kubernetes.io/projected/e3f3e38b-6d52-4beb-886f-ad3ddf93008f-kube-api-access-dsdwm\") pod \"e3f3e38b-6d52-4beb-886f-ad3ddf93008f\" (UID: \"e3f3e38b-6d52-4beb-886f-ad3ddf93008f\") "
Sep 10 17:19:39.898491 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.898418    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3f3e38b-6d52-4beb-886f-ad3ddf93008f-util\") pod \"e3f3e38b-6d52-4beb-886f-ad3ddf93008f\" (UID: \"e3f3e38b-6d52-4beb-886f-ad3ddf93008f\") "
Sep 10 17:19:39.898603 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.898492    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr8fm\" (UniqueName: \"kubernetes.io/projected/8e21d82f-ef27-4e57-8556-3aba57c3e09d-kube-api-access-cr8fm\") pod \"cert-manager-cainjector-559cdd8c87-kvn7s\" (UID: \"8e21d82f-ef27-4e57-8556-3aba57c3e09d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kvn7s"
Sep 10 17:19:39.898603 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.898552    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8e21d82f-ef27-4e57-8556-3aba57c3e09d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kvn7s\" (UID: \"8e21d82f-ef27-4e57-8556-3aba57c3e09d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kvn7s"
Sep 10 17:19:39.898748 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.898715    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3f3e38b-6d52-4beb-886f-ad3ddf93008f-bundle" (OuterVolumeSpecName: "bundle") pod "e3f3e38b-6d52-4beb-886f-ad3ddf93008f" (UID: "e3f3e38b-6d52-4beb-886f-ad3ddf93008f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:19:39.900490 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.900468    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3f3e38b-6d52-4beb-886f-ad3ddf93008f-kube-api-access-dsdwm" (OuterVolumeSpecName: "kube-api-access-dsdwm") pod "e3f3e38b-6d52-4beb-886f-ad3ddf93008f" (UID: "e3f3e38b-6d52-4beb-886f-ad3ddf93008f"). InnerVolumeSpecName "kube-api-access-dsdwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:19:39.902500 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.902481    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3f3e38b-6d52-4beb-886f-ad3ddf93008f-util" (OuterVolumeSpecName: "util") pod "e3f3e38b-6d52-4beb-886f-ad3ddf93008f" (UID: "e3f3e38b-6d52-4beb-886f-ad3ddf93008f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:19:39.909498 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.909478    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr8fm\" (UniqueName: \"kubernetes.io/projected/8e21d82f-ef27-4e57-8556-3aba57c3e09d-kube-api-access-cr8fm\") pod \"cert-manager-cainjector-559cdd8c87-kvn7s\" (UID: \"8e21d82f-ef27-4e57-8556-3aba57c3e09d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kvn7s"
Sep 10 17:19:39.911666 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.911640    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8e21d82f-ef27-4e57-8556-3aba57c3e09d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kvn7s\" (UID: \"8e21d82f-ef27-4e57-8556-3aba57c3e09d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kvn7s"
Sep 10 17:19:39.988811 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.988785    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kvn7s"
Sep 10 17:19:39.999195 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.999172    2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3f3e38b-6d52-4beb-886f-ad3ddf93008f-bundle\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:19:39.999287 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.999199    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsdwm\" (UniqueName: \"kubernetes.io/projected/e3f3e38b-6d52-4beb-886f-ad3ddf93008f-kube-api-access-dsdwm\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:19:39.999287 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:39.999215    2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3f3e38b-6d52-4beb-886f-ad3ddf93008f-util\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:19:40.279969 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:40.279935    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kvn7s"]
Sep 10 17:19:40.282866 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:19:40.282838    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e21d82f_ef27_4e57_8556_3aba57c3e09d.slice/crio-d96bb7a5ac27d61d2213457a0ed164a53975978c72b5c2ad1f986223e6ef025e WatchSource:0}: Error finding container d96bb7a5ac27d61d2213457a0ed164a53975978c72b5c2ad1f986223e6ef025e: Status 404 returned error can't find the container with id d96bb7a5ac27d61d2213457a0ed164a53975978c72b5c2ad1f986223e6ef025e
Sep 10 17:19:40.633076 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:40.632988    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx" event={"ID":"e3f3e38b-6d52-4beb-886f-ad3ddf93008f","Type":"ContainerDied","Data":"bcbb6801e0b9c78822f4e690629305e0d11d5d0e5d4b858b92dcdf0b6ea3b2fd"}
Sep 10 17:19:40.633076 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:40.633067    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bcbb6801e0b9c78822f4e690629305e0d11d5d0e5d4b858b92dcdf0b6ea3b2fd"
Sep 10 17:19:40.633076 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:40.632994    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffnhjx"
Sep 10 17:19:40.634843 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:40.634816    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kvn7s" event={"ID":"8e21d82f-ef27-4e57-8556-3aba57c3e09d","Type":"ContainerStarted","Data":"88e026b31741ca08cd801bcb19789c91452b7a834ae1d97e2a79635c7145e7a2"}
Sep 10 17:19:40.634936 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:40.634855    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kvn7s" event={"ID":"8e21d82f-ef27-4e57-8556-3aba57c3e09d","Type":"ContainerStarted","Data":"d96bb7a5ac27d61d2213457a0ed164a53975978c72b5c2ad1f986223e6ef025e"}
Sep 10 17:19:40.655275 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:40.655222    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kvn7s" podStartSLOduration=1.655203451 podStartE2EDuration="1.655203451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:19:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:19:40.654377785 +0000 UTC m=+384.335550732" watchObservedRunningTime="2026-09-10 17:19:40.655203451 +0000 UTC m=+384.336376409"
Sep 10 17:19:41.312016 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:41.311979    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:41.467811 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:41.467772    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:43.502631 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:43.502601    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:43.617250 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:43.617223    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-55gq5"
Sep 10 17:19:43.645652 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:43.645623    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:44.184054 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:44.184015    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:48.306642 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:48.306599    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:48.403261 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:48.403229    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:53.816524 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:53.816486    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:53.888650 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:53.888612    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z7lxt"]
Sep 10 17:19:53.888962 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:53.888949    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3f3e38b-6d52-4beb-886f-ad3ddf93008f" containerName="pull"
Sep 10 17:19:53.889011 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:53.888963    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3f3e38b-6d52-4beb-886f-ad3ddf93008f" containerName="pull"
Sep 10 17:19:53.889011 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:53.888974    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3f3e38b-6d52-4beb-886f-ad3ddf93008f" containerName="util"
Sep 10 17:19:53.889011 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:53.888979    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3f3e38b-6d52-4beb-886f-ad3ddf93008f" containerName="util"
Sep 10 17:19:53.889011 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:53.888991    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3f3e38b-6d52-4beb-886f-ad3ddf93008f" containerName="extract"
Sep 10 17:19:53.889011 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:53.888996    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3f3e38b-6d52-4beb-886f-ad3ddf93008f" containerName="extract"
Sep 10 17:19:53.889170 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:53.889054    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3f3e38b-6d52-4beb-886f-ad3ddf93008f" containerName="extract"
Sep 10 17:19:53.910464 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:53.910442    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-z7lxt"
Sep 10 17:19:53.913394 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:53.913376    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-84qw4\""
Sep 10 17:19:53.914540 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:53.914510    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z7lxt"]
Sep 10 17:19:53.945867 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:53.945841    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:54.015756 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:54.015725    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d2ee7517-9671-440b-b3d4-f011deafb54d-bound-sa-token\") pod \"cert-manager-55b4d954d5-z7lxt\" (UID: \"d2ee7517-9671-440b-b3d4-f011deafb54d\") " pod="cert-manager/cert-manager-55b4d954d5-z7lxt"
Sep 10 17:19:54.015885 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:54.015764    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cntj\" (UniqueName: \"kubernetes.io/projected/d2ee7517-9671-440b-b3d4-f011deafb54d-kube-api-access-2cntj\") pod \"cert-manager-55b4d954d5-z7lxt\" (UID: \"d2ee7517-9671-440b-b3d4-f011deafb54d\") " pod="cert-manager/cert-manager-55b4d954d5-z7lxt"
Sep 10 17:19:54.116772 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:54.116687    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d2ee7517-9671-440b-b3d4-f011deafb54d-bound-sa-token\") pod \"cert-manager-55b4d954d5-z7lxt\" (UID: \"d2ee7517-9671-440b-b3d4-f011deafb54d\") " pod="cert-manager/cert-manager-55b4d954d5-z7lxt"
Sep 10 17:19:54.116772 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:54.116735    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cntj\" (UniqueName: \"kubernetes.io/projected/d2ee7517-9671-440b-b3d4-f011deafb54d-kube-api-access-2cntj\") pod \"cert-manager-55b4d954d5-z7lxt\" (UID: \"d2ee7517-9671-440b-b3d4-f011deafb54d\") " pod="cert-manager/cert-manager-55b4d954d5-z7lxt"
Sep 10 17:19:54.126839 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:54.126807    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d2ee7517-9671-440b-b3d4-f011deafb54d-bound-sa-token\") pod \"cert-manager-55b4d954d5-z7lxt\" (UID: \"d2ee7517-9671-440b-b3d4-f011deafb54d\") " pod="cert-manager/cert-manager-55b4d954d5-z7lxt"
Sep 10 17:19:54.130583 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:54.130558    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cntj\" (UniqueName: \"kubernetes.io/projected/d2ee7517-9671-440b-b3d4-f011deafb54d-kube-api-access-2cntj\") pod \"cert-manager-55b4d954d5-z7lxt\" (UID: \"d2ee7517-9671-440b-b3d4-f011deafb54d\") " pod="cert-manager/cert-manager-55b4d954d5-z7lxt"
Sep 10 17:19:54.221630 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:54.221599    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-z7lxt"
Sep 10 17:19:54.521276 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:54.521247    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z7lxt"]
Sep 10 17:19:54.525796 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:19:54.525762    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2ee7517_9671_440b_b3d4_f011deafb54d.slice/crio-e9ea3eb9fb2c2ba38830d48c40e361489991a0d19af7c568743c7d00885079ba WatchSource:0}: Error finding container e9ea3eb9fb2c2ba38830d48c40e361489991a0d19af7c568743c7d00885079ba: Status 404 returned error can't find the container with id e9ea3eb9fb2c2ba38830d48c40e361489991a0d19af7c568743c7d00885079ba
Sep 10 17:19:54.697368 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:54.697337    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-z7lxt" event={"ID":"d2ee7517-9671-440b-b3d4-f011deafb54d","Type":"ContainerStarted","Data":"8619be2a5f75ff95ce9fb72de6a2cc6264174bae27a0385bf3ad03094c667a58"}
Sep 10 17:19:54.697368 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:54.697373    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-z7lxt" event={"ID":"d2ee7517-9671-440b-b3d4-f011deafb54d","Type":"ContainerStarted","Data":"e9ea3eb9fb2c2ba38830d48c40e361489991a0d19af7c568743c7d00885079ba"}
Sep 10 17:19:54.732878 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:54.732823    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-z7lxt" podStartSLOduration=1.73280418 podStartE2EDuration="1.73280418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:19:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:19:54.722772727 +0000 UTC m=+398.403945668" watchObservedRunningTime="2026-09-10 17:19:54.73280418 +0000 UTC m=+398.413977126"
Sep 10 17:19:55.786332 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:55.786294    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz"]
Sep 10 17:19:55.813821 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:55.813795    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz"]
Sep 10 17:19:55.813971 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:55.813882    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz"
Sep 10 17:19:55.817361 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:55.817335    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 17:19:55.817651 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:55.817634    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 17:19:55.817761 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:55.817685    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9gr7t\""
Sep 10 17:19:55.909553 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:55.909525    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:19:55.933524 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:55.933490    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/820a5820-c70f-49ce-83b5-ab1b8afc4268-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz\" (UID: \"820a5820-c70f-49ce-83b5-ab1b8afc4268\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz"
Sep 10 17:19:55.933660 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:55.933581    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/820a5820-c70f-49ce-83b5-ab1b8afc4268-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz\" (UID: \"820a5820-c70f-49ce-83b5-ab1b8afc4268\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz"
Sep 10 17:19:55.933660 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:55.933620    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcw2h\" (UniqueName: \"kubernetes.io/projected/820a5820-c70f-49ce-83b5-ab1b8afc4268-kube-api-access-gcw2h\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz\" (UID: \"820a5820-c70f-49ce-83b5-ab1b8afc4268\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz"
Sep 10 17:19:56.034666 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:56.034634    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/820a5820-c70f-49ce-83b5-ab1b8afc4268-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz\" (UID: \"820a5820-c70f-49ce-83b5-ab1b8afc4268\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz"
Sep 10 17:19:56.034810 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:56.034703    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/820a5820-c70f-49ce-83b5-ab1b8afc4268-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz\" (UID: \"820a5820-c70f-49ce-83b5-ab1b8afc4268\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz"
Sep 10 17:19:56.034810 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:56.034742    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcw2h\" (UniqueName: \"kubernetes.io/projected/820a5820-c70f-49ce-83b5-ab1b8afc4268-kube-api-access-gcw2h\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz\" (UID: \"820a5820-c70f-49ce-83b5-ab1b8afc4268\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz"
Sep 10 17:19:56.035023 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:56.035001    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/820a5820-c70f-49ce-83b5-ab1b8afc4268-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz\" (UID: \"820a5820-c70f-49ce-83b5-ab1b8afc4268\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz"
Sep 10 17:19:56.035089 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:56.035031    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/820a5820-c70f-49ce-83b5-ab1b8afc4268-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz\" (UID: \"820a5820-c70f-49ce-83b5-ab1b8afc4268\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz"
Sep 10 17:19:56.043990 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:56.043934    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcw2h\" (UniqueName: \"kubernetes.io/projected/820a5820-c70f-49ce-83b5-ab1b8afc4268-kube-api-access-gcw2h\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz\" (UID: \"820a5820-c70f-49ce-83b5-ab1b8afc4268\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz"
Sep 10 17:19:56.123262 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:56.123231    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz"
Sep 10 17:19:56.413560 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:56.413483    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz"]
Sep 10 17:19:56.417440 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:19:56.417412    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod820a5820_c70f_49ce_83b5_ab1b8afc4268.slice/crio-dbe9036da3ed320108c09e4fa83c2fb8de60dec548d0857efd5f26c8d7da4237 WatchSource:0}: Error finding container dbe9036da3ed320108c09e4fa83c2fb8de60dec548d0857efd5f26c8d7da4237: Status 404 returned error can't find the container with id dbe9036da3ed320108c09e4fa83c2fb8de60dec548d0857efd5f26c8d7da4237
Sep 10 17:19:56.708430 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:56.708344    2566 generic.go:356] "Generic (PLEG): container finished" podID="820a5820-c70f-49ce-83b5-ab1b8afc4268" containerID="09282d67238ee65252cffd49d779352c34fc711c240ad8fc4d53aa41427193f8" exitCode=0
Sep 10 17:19:56.708550 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:56.708426    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz" event={"ID":"820a5820-c70f-49ce-83b5-ab1b8afc4268","Type":"ContainerDied","Data":"09282d67238ee65252cffd49d779352c34fc711c240ad8fc4d53aa41427193f8"}
Sep 10 17:19:56.708550 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:56.708485    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz" event={"ID":"820a5820-c70f-49ce-83b5-ab1b8afc4268","Type":"ContainerStarted","Data":"dbe9036da3ed320108c09e4fa83c2fb8de60dec548d0857efd5f26c8d7da4237"}
Sep 10 17:19:57.715224 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:57.715140    2566 generic.go:356] "Generic (PLEG): container finished" podID="820a5820-c70f-49ce-83b5-ab1b8afc4268" containerID="a147b2cb0499be8f863635815d15219f370b6e6bc091059a59d10b72019eb88a" exitCode=0
Sep 10 17:19:57.715224 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:57.715153    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz" event={"ID":"820a5820-c70f-49ce-83b5-ab1b8afc4268","Type":"ContainerDied","Data":"a147b2cb0499be8f863635815d15219f370b6e6bc091059a59d10b72019eb88a"}
Sep 10 17:19:58.721590 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:58.721553    2566 generic.go:356] "Generic (PLEG): container finished" podID="820a5820-c70f-49ce-83b5-ab1b8afc4268" containerID="b85be28ca308c7d708f0310a4a2d1422c647bdbaaa4a4d38dbf70ac36a025be5" exitCode=0
Sep 10 17:19:58.721941 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:58.721636    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz" event={"ID":"820a5820-c70f-49ce-83b5-ab1b8afc4268","Type":"ContainerDied","Data":"b85be28ca308c7d708f0310a4a2d1422c647bdbaaa4a4d38dbf70ac36a025be5"}
Sep 10 17:19:59.848338 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:59.848308    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz"
Sep 10 17:19:59.969273 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:59.969244    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/820a5820-c70f-49ce-83b5-ab1b8afc4268-util\") pod \"820a5820-c70f-49ce-83b5-ab1b8afc4268\" (UID: \"820a5820-c70f-49ce-83b5-ab1b8afc4268\") "
Sep 10 17:19:59.969414 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:59.969320    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcw2h\" (UniqueName: \"kubernetes.io/projected/820a5820-c70f-49ce-83b5-ab1b8afc4268-kube-api-access-gcw2h\") pod \"820a5820-c70f-49ce-83b5-ab1b8afc4268\" (UID: \"820a5820-c70f-49ce-83b5-ab1b8afc4268\") "
Sep 10 17:19:59.969414 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:59.969366    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/820a5820-c70f-49ce-83b5-ab1b8afc4268-bundle\") pod \"820a5820-c70f-49ce-83b5-ab1b8afc4268\" (UID: \"820a5820-c70f-49ce-83b5-ab1b8afc4268\") "
Sep 10 17:19:59.970087 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:59.970059    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/820a5820-c70f-49ce-83b5-ab1b8afc4268-bundle" (OuterVolumeSpecName: "bundle") pod "820a5820-c70f-49ce-83b5-ab1b8afc4268" (UID: "820a5820-c70f-49ce-83b5-ab1b8afc4268"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:19:59.971322 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:59.971296    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/820a5820-c70f-49ce-83b5-ab1b8afc4268-kube-api-access-gcw2h" (OuterVolumeSpecName: "kube-api-access-gcw2h") pod "820a5820-c70f-49ce-83b5-ab1b8afc4268" (UID: "820a5820-c70f-49ce-83b5-ab1b8afc4268"). InnerVolumeSpecName "kube-api-access-gcw2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:19:59.974764 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:19:59.974722    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/820a5820-c70f-49ce-83b5-ab1b8afc4268-util" (OuterVolumeSpecName: "util") pod "820a5820-c70f-49ce-83b5-ab1b8afc4268" (UID: "820a5820-c70f-49ce-83b5-ab1b8afc4268"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:20:00.070089 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:00.070057    2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/820a5820-c70f-49ce-83b5-ab1b8afc4268-util\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:20:00.070089 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:00.070084    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcw2h\" (UniqueName: \"kubernetes.io/projected/820a5820-c70f-49ce-83b5-ab1b8afc4268-kube-api-access-gcw2h\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:20:00.070089 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:00.070094    2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/820a5820-c70f-49ce-83b5-ab1b8afc4268-bundle\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:20:00.732438 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:00.732406    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz" event={"ID":"820a5820-c70f-49ce-83b5-ab1b8afc4268","Type":"ContainerDied","Data":"dbe9036da3ed320108c09e4fa83c2fb8de60dec548d0857efd5f26c8d7da4237"}
Sep 10 17:20:00.732600 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:00.732442    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbj8dz"
Sep 10 17:20:00.732600 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:00.732478    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dbe9036da3ed320108c09e4fa83c2fb8de60dec548d0857efd5f26c8d7da4237"
Sep 10 17:20:11.574345 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:11.574313    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:11.733216 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:11.733177    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:12.312904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:12.312868    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:12.843612 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:12.843581    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:12.947613 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:12.947579    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:13.031682 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.031647    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh"]
Sep 10 17:20:13.031993 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.031980    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="820a5820-c70f-49ce-83b5-ab1b8afc4268" containerName="pull"
Sep 10 17:20:13.032037 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.031995    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="820a5820-c70f-49ce-83b5-ab1b8afc4268" containerName="pull"
Sep 10 17:20:13.032037 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.032011    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="820a5820-c70f-49ce-83b5-ab1b8afc4268" containerName="extract"
Sep 10 17:20:13.032037 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.032017    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="820a5820-c70f-49ce-83b5-ab1b8afc4268" containerName="extract"
Sep 10 17:20:13.032037 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.032025    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="820a5820-c70f-49ce-83b5-ab1b8afc4268" containerName="util"
Sep 10 17:20:13.032037 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.032031    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="820a5820-c70f-49ce-83b5-ab1b8afc4268" containerName="util"
Sep 10 17:20:13.032212 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.032084    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="820a5820-c70f-49ce-83b5-ab1b8afc4268" containerName="extract"
Sep 10 17:20:13.035581 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.035560    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh"
Sep 10 17:20:13.038839 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.038806    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9gr7t\""
Sep 10 17:20:13.038839 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.038806    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 17:20:13.039676 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.039658    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 17:20:13.055832 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.055801    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"]
Sep 10 17:20:13.059878 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.059857    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh"]
Sep 10 17:20:13.059986 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.059958    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:13.065100 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.065078    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 17:20:13.065771 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.065535    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 17:20:13.065771 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.065561    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 17:20:13.065771 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.065535    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xgkxt\""
Sep 10 17:20:13.065771 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.065626    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 17:20:13.075136 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.075095    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f13ffffc-cdeb-490c-ba71-de579b6195ff-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh\" (UID: \"f13ffffc-cdeb-490c-ba71-de579b6195ff\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh"
Sep 10 17:20:13.075231 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.075161    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f13ffffc-cdeb-490c-ba71-de579b6195ff-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh\" (UID: \"f13ffffc-cdeb-490c-ba71-de579b6195ff\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh"
Sep 10 17:20:13.075231 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.075194    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x2sk\" (UniqueName: \"kubernetes.io/projected/c4aa8ddd-f642-4ce0-b313-8684380b650f-kube-api-access-5x2sk\") pod \"opendatahub-operator-controller-manager-8dc495f6d-rt6q6\" (UID: \"c4aa8ddd-f642-4ce0-b313-8684380b650f\") " pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:13.075231 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.075226    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c4aa8ddd-f642-4ce0-b313-8684380b650f-webhook-cert\") pod \"opendatahub-operator-controller-manager-8dc495f6d-rt6q6\" (UID: \"c4aa8ddd-f642-4ce0-b313-8684380b650f\") " pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:13.075334 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.075266    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c4aa8ddd-f642-4ce0-b313-8684380b650f-manifests\") pod \"opendatahub-operator-controller-manager-8dc495f6d-rt6q6\" (UID: \"c4aa8ddd-f642-4ce0-b313-8684380b650f\") " pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:13.075367 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.075336    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4aa8ddd-f642-4ce0-b313-8684380b650f-tmp\") pod \"opendatahub-operator-controller-manager-8dc495f6d-rt6q6\" (UID: \"c4aa8ddd-f642-4ce0-b313-8684380b650f\") " pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:13.075410 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.075375    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c4aa8ddd-f642-4ce0-b313-8684380b650f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8dc495f6d-rt6q6\" (UID: \"c4aa8ddd-f642-4ce0-b313-8684380b650f\") " pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:13.075410 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.075402    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbmkr\" (UniqueName: \"kubernetes.io/projected/f13ffffc-cdeb-490c-ba71-de579b6195ff-kube-api-access-tbmkr\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh\" (UID: \"f13ffffc-cdeb-490c-ba71-de579b6195ff\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh"
Sep 10 17:20:13.100543 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.100462    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"]
Sep 10 17:20:13.176740 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.176709    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c4aa8ddd-f642-4ce0-b313-8684380b650f-webhook-cert\") pod \"opendatahub-operator-controller-manager-8dc495f6d-rt6q6\" (UID: \"c4aa8ddd-f642-4ce0-b313-8684380b650f\") " pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:13.176917 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.176775    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c4aa8ddd-f642-4ce0-b313-8684380b650f-manifests\") pod \"opendatahub-operator-controller-manager-8dc495f6d-rt6q6\" (UID: \"c4aa8ddd-f642-4ce0-b313-8684380b650f\") " pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:13.176917 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.176818    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4aa8ddd-f642-4ce0-b313-8684380b650f-tmp\") pod \"opendatahub-operator-controller-manager-8dc495f6d-rt6q6\" (UID: \"c4aa8ddd-f642-4ce0-b313-8684380b650f\") " pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:13.176917 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.176844    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c4aa8ddd-f642-4ce0-b313-8684380b650f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8dc495f6d-rt6q6\" (UID: \"c4aa8ddd-f642-4ce0-b313-8684380b650f\") " pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:13.176917 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.176873    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbmkr\" (UniqueName: \"kubernetes.io/projected/f13ffffc-cdeb-490c-ba71-de579b6195ff-kube-api-access-tbmkr\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh\" (UID: \"f13ffffc-cdeb-490c-ba71-de579b6195ff\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh"
Sep 10 17:20:13.177176 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.176939    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f13ffffc-cdeb-490c-ba71-de579b6195ff-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh\" (UID: \"f13ffffc-cdeb-490c-ba71-de579b6195ff\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh"
Sep 10 17:20:13.177176 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.176992    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f13ffffc-cdeb-490c-ba71-de579b6195ff-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh\" (UID: \"f13ffffc-cdeb-490c-ba71-de579b6195ff\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh"
Sep 10 17:20:13.177176 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.177032    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x2sk\" (UniqueName: \"kubernetes.io/projected/c4aa8ddd-f642-4ce0-b313-8684380b650f-kube-api-access-5x2sk\") pod \"opendatahub-operator-controller-manager-8dc495f6d-rt6q6\" (UID: \"c4aa8ddd-f642-4ce0-b313-8684380b650f\") " pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:13.177337 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.177314    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c4aa8ddd-f642-4ce0-b313-8684380b650f-manifests\") pod \"opendatahub-operator-controller-manager-8dc495f6d-rt6q6\" (UID: \"c4aa8ddd-f642-4ce0-b313-8684380b650f\") " pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:13.177416 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.177394    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f13ffffc-cdeb-490c-ba71-de579b6195ff-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh\" (UID: \"f13ffffc-cdeb-490c-ba71-de579b6195ff\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh"
Sep 10 17:20:13.177544 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.177459    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f13ffffc-cdeb-490c-ba71-de579b6195ff-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh\" (UID: \"f13ffffc-cdeb-490c-ba71-de579b6195ff\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh"
Sep 10 17:20:13.179205 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.179185    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4aa8ddd-f642-4ce0-b313-8684380b650f-tmp\") pod \"opendatahub-operator-controller-manager-8dc495f6d-rt6q6\" (UID: \"c4aa8ddd-f642-4ce0-b313-8684380b650f\") " pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:13.179356 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.179336    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c4aa8ddd-f642-4ce0-b313-8684380b650f-webhook-cert\") pod \"opendatahub-operator-controller-manager-8dc495f6d-rt6q6\" (UID: \"c4aa8ddd-f642-4ce0-b313-8684380b650f\") " pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:13.179516 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.179499    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c4aa8ddd-f642-4ce0-b313-8684380b650f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8dc495f6d-rt6q6\" (UID: \"c4aa8ddd-f642-4ce0-b313-8684380b650f\") " pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:13.191433 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.191412    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x2sk\" (UniqueName: \"kubernetes.io/projected/c4aa8ddd-f642-4ce0-b313-8684380b650f-kube-api-access-5x2sk\") pod \"opendatahub-operator-controller-manager-8dc495f6d-rt6q6\" (UID: \"c4aa8ddd-f642-4ce0-b313-8684380b650f\") " pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:13.192719 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.192702    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbmkr\" (UniqueName: \"kubernetes.io/projected/f13ffffc-cdeb-490c-ba71-de579b6195ff-kube-api-access-tbmkr\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh\" (UID: \"f13ffffc-cdeb-490c-ba71-de579b6195ff\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh"
Sep 10 17:20:13.299988 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.299954    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:13.344990 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.344961    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh"
Sep 10 17:20:13.370989 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.370958    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:13.569766 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.569725    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:13.643437 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.643412    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh"]
Sep 10 17:20:13.645782 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:20:13.645756    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13ffffc_cdeb_490c_ba71_de579b6195ff.slice/crio-48453f6fc4d394c7e28938bdff5daa2d813d6d23dc0df3a12915d94788064e25 WatchSource:0}: Error finding container 48453f6fc4d394c7e28938bdff5daa2d813d6d23dc0df3a12915d94788064e25: Status 404 returned error can't find the container with id 48453f6fc4d394c7e28938bdff5daa2d813d6d23dc0df3a12915d94788064e25
Sep 10 17:20:13.687312 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.687217    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"]
Sep 10 17:20:13.689946 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:20:13.689919    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4aa8ddd_f642_4ce0_b313_8684380b650f.slice/crio-655d95fdfa0ecdeca67da2e5d3247c2ee95921663b98a1b8f30f7086134d2171 WatchSource:0}: Error finding container 655d95fdfa0ecdeca67da2e5d3247c2ee95921663b98a1b8f30f7086134d2171: Status 404 returned error can't find the container with id 655d95fdfa0ecdeca67da2e5d3247c2ee95921663b98a1b8f30f7086134d2171
Sep 10 17:20:13.741891 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.741858    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:13.794938 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.794911    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6" event={"ID":"c4aa8ddd-f642-4ce0-b313-8684380b650f","Type":"ContainerStarted","Data":"655d95fdfa0ecdeca67da2e5d3247c2ee95921663b98a1b8f30f7086134d2171"}
Sep 10 17:20:13.796418 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.796389    2566 generic.go:356] "Generic (PLEG): container finished" podID="f13ffffc-cdeb-490c-ba71-de579b6195ff" containerID="2d964d4edc915fe24e4a261abf224312729b80f4d5e2a266e81e72785a19a5cc" exitCode=0
Sep 10 17:20:13.796529 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.796429    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh" event={"ID":"f13ffffc-cdeb-490c-ba71-de579b6195ff","Type":"ContainerDied","Data":"2d964d4edc915fe24e4a261abf224312729b80f4d5e2a266e81e72785a19a5cc"}
Sep 10 17:20:13.796529 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.796466    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh" event={"ID":"f13ffffc-cdeb-490c-ba71-de579b6195ff","Type":"ContainerStarted","Data":"48453f6fc4d394c7e28938bdff5daa2d813d6d23dc0df3a12915d94788064e25"}
Sep 10 17:20:13.882528 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:13.882460    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:14.037760 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:14.037728    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:14.171228 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:14.171169    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:14.804926 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:14.804891    2566 generic.go:356] "Generic (PLEG): container finished" podID="f13ffffc-cdeb-490c-ba71-de579b6195ff" containerID="9d4fc02ddb8ec2f2e3261a206b9bc3521776cb9ccd8c63beff3b64ebf11e49a2" exitCode=0
Sep 10 17:20:14.805145 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:14.804977    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh" event={"ID":"f13ffffc-cdeb-490c-ba71-de579b6195ff","Type":"ContainerDied","Data":"9d4fc02ddb8ec2f2e3261a206b9bc3521776cb9ccd8c63beff3b64ebf11e49a2"}
Sep 10 17:20:15.813239 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:15.813198    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh" event={"ID":"f13ffffc-cdeb-490c-ba71-de579b6195ff","Type":"ContainerStarted","Data":"c5fabd891be9397cbd23e1fa91db17ce47e2ca5944435c00bd947d8f0ae7f69d"}
Sep 10 17:20:15.837101 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:15.837049    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh" podStartSLOduration=3.195103815 podStartE2EDuration="3.837030288s" totalImagesPullingTime="641.926473ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:20:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:20:15.836939707 +0000 UTC m=+419.518112654" watchObservedRunningTime="2026-09-10 17:20:15.837030288 +0000 UTC m=+419.518203231"
Sep 10 17:20:16.820547 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:16.820466    2566 generic.go:356] "Generic (PLEG): container finished" podID="f13ffffc-cdeb-490c-ba71-de579b6195ff" containerID="c5fabd891be9397cbd23e1fa91db17ce47e2ca5944435c00bd947d8f0ae7f69d" exitCode=0
Sep 10 17:20:16.820950 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:16.820538    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh" event={"ID":"f13ffffc-cdeb-490c-ba71-de579b6195ff","Type":"ContainerDied","Data":"c5fabd891be9397cbd23e1fa91db17ce47e2ca5944435c00bd947d8f0ae7f69d"}
Sep 10 17:20:16.822261 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:16.822236    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6" event={"ID":"c4aa8ddd-f642-4ce0-b313-8684380b650f","Type":"ContainerStarted","Data":"f4902dce60c2ff9b330d76a89545f051053c841705cfc001a325494665400c27"}
Sep 10 17:20:17.828041 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:17.828010    2566 generic.go:356] "Generic (PLEG): container finished" podID="c4aa8ddd-f642-4ce0-b313-8684380b650f" containerID="f4902dce60c2ff9b330d76a89545f051053c841705cfc001a325494665400c27" exitCode=0
Sep 10 17:20:17.828379 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:17.828062    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6" event={"ID":"c4aa8ddd-f642-4ce0-b313-8684380b650f","Type":"ContainerDied","Data":"f4902dce60c2ff9b330d76a89545f051053c841705cfc001a325494665400c27"}
Sep 10 17:20:17.967617 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:17.967591    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh"
Sep 10 17:20:18.020814 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:18.020790    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f13ffffc-cdeb-490c-ba71-de579b6195ff-bundle\") pod \"f13ffffc-cdeb-490c-ba71-de579b6195ff\" (UID: \"f13ffffc-cdeb-490c-ba71-de579b6195ff\") "
Sep 10 17:20:18.020990 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:18.020821    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbmkr\" (UniqueName: \"kubernetes.io/projected/f13ffffc-cdeb-490c-ba71-de579b6195ff-kube-api-access-tbmkr\") pod \"f13ffffc-cdeb-490c-ba71-de579b6195ff\" (UID: \"f13ffffc-cdeb-490c-ba71-de579b6195ff\") "
Sep 10 17:20:18.020990 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:18.020894    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f13ffffc-cdeb-490c-ba71-de579b6195ff-util\") pod \"f13ffffc-cdeb-490c-ba71-de579b6195ff\" (UID: \"f13ffffc-cdeb-490c-ba71-de579b6195ff\") "
Sep 10 17:20:18.021534 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:18.021508    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f13ffffc-cdeb-490c-ba71-de579b6195ff-bundle" (OuterVolumeSpecName: "bundle") pod "f13ffffc-cdeb-490c-ba71-de579b6195ff" (UID: "f13ffffc-cdeb-490c-ba71-de579b6195ff"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:20:18.022777 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:18.022754    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f13ffffc-cdeb-490c-ba71-de579b6195ff-kube-api-access-tbmkr" (OuterVolumeSpecName: "kube-api-access-tbmkr") pod "f13ffffc-cdeb-490c-ba71-de579b6195ff" (UID: "f13ffffc-cdeb-490c-ba71-de579b6195ff"). InnerVolumeSpecName "kube-api-access-tbmkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:20:18.028072 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:18.028045    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f13ffffc-cdeb-490c-ba71-de579b6195ff-util" (OuterVolumeSpecName: "util") pod "f13ffffc-cdeb-490c-ba71-de579b6195ff" (UID: "f13ffffc-cdeb-490c-ba71-de579b6195ff"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:20:18.121987 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:18.121949    2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f13ffffc-cdeb-490c-ba71-de579b6195ff-util\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:20:18.121987 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:18.121979    2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f13ffffc-cdeb-490c-ba71-de579b6195ff-bundle\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:20:18.121987 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:18.121990    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbmkr\" (UniqueName: \"kubernetes.io/projected/f13ffffc-cdeb-490c-ba71-de579b6195ff-kube-api-access-tbmkr\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:20:18.835835 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:18.835804    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh"
Sep 10 17:20:18.835835 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:18.835827    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h7dlh" event={"ID":"f13ffffc-cdeb-490c-ba71-de579b6195ff","Type":"ContainerDied","Data":"48453f6fc4d394c7e28938bdff5daa2d813d6d23dc0df3a12915d94788064e25"}
Sep 10 17:20:18.836399 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:18.835877    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48453f6fc4d394c7e28938bdff5daa2d813d6d23dc0df3a12915d94788064e25"
Sep 10 17:20:18.838227 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:18.838188    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6" event={"ID":"c4aa8ddd-f642-4ce0-b313-8684380b650f","Type":"ContainerStarted","Data":"0864802b8f23775c907ce734b7d4a91e4c67d9aeb47a8cd6b5551491c8e7f26f"}
Sep 10 17:20:18.838370 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:18.838330    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:18.889251 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:18.889210    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6" podStartSLOduration=1.583996243 podStartE2EDuration="5.889192612s" totalImagesPullingTime="3.241325022s" totalInitContainerRuntime="1.063871347s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:20:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:20:18.887550916 +0000 UTC m=+422.568723859" watchObservedRunningTime="2026-09-10 17:20:18.889192612 +0000 UTC m=+422.570365558"
Sep 10 17:20:28.795060 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:28.795013    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:28.915182 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:28.915149    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:29.247304 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:29.247268    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:29.356859 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:29.356825    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:29.845174 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:29.845142    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8dc495f6d-rt6q6"
Sep 10 17:20:30.190532 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:30.190437    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:30.290366 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:30.290338    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:31.419676 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.419638    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:31.636877 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.636845    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"]
Sep 10 17:20:31.637216 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.637203    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13ffffc-cdeb-490c-ba71-de579b6195ff" containerName="util"
Sep 10 17:20:31.637265 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.637217    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13ffffc-cdeb-490c-ba71-de579b6195ff" containerName="util"
Sep 10 17:20:31.637265 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.637228    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13ffffc-cdeb-490c-ba71-de579b6195ff" containerName="extract"
Sep 10 17:20:31.637265 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.637234    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13ffffc-cdeb-490c-ba71-de579b6195ff" containerName="extract"
Sep 10 17:20:31.637265 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.637243    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13ffffc-cdeb-490c-ba71-de579b6195ff" containerName="pull"
Sep 10 17:20:31.637265 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.637249    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13ffffc-cdeb-490c-ba71-de579b6195ff" containerName="pull"
Sep 10 17:20:31.637420 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.637326    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13ffffc-cdeb-490c-ba71-de579b6195ff" containerName="extract"
Sep 10 17:20:31.645169 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.645151    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"
Sep 10 17:20:31.664146 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.664112    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 17:20:31.664300 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.664283    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 17:20:31.664603 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.664592    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ll68g\""
Sep 10 17:20:31.664646 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.664628    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 17:20:31.665255 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.665242    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 17:20:31.665298 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.665244    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 17:20:31.675999 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.675952    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"]
Sep 10 17:20:31.692671 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.692644    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:31.735791 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.735759    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/69b6f0c6-12c7-45b7-92c0-9615b59c9afb-manager-config\") pod \"lws-controller-manager-8655464fff-lvz59\" (UID: \"69b6f0c6-12c7-45b7-92c0-9615b59c9afb\") " pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"
Sep 10 17:20:31.735943 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.735924    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6ccf\" (UniqueName: \"kubernetes.io/projected/69b6f0c6-12c7-45b7-92c0-9615b59c9afb-kube-api-access-s6ccf\") pod \"lws-controller-manager-8655464fff-lvz59\" (UID: \"69b6f0c6-12c7-45b7-92c0-9615b59c9afb\") " pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"
Sep 10 17:20:31.736003 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.735957    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69b6f0c6-12c7-45b7-92c0-9615b59c9afb-cert\") pod \"lws-controller-manager-8655464fff-lvz59\" (UID: \"69b6f0c6-12c7-45b7-92c0-9615b59c9afb\") " pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"
Sep 10 17:20:31.736003 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.735975    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/69b6f0c6-12c7-45b7-92c0-9615b59c9afb-metrics-cert\") pod \"lws-controller-manager-8655464fff-lvz59\" (UID: \"69b6f0c6-12c7-45b7-92c0-9615b59c9afb\") " pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"
Sep 10 17:20:31.836878 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.836851    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6ccf\" (UniqueName: \"kubernetes.io/projected/69b6f0c6-12c7-45b7-92c0-9615b59c9afb-kube-api-access-s6ccf\") pod \"lws-controller-manager-8655464fff-lvz59\" (UID: \"69b6f0c6-12c7-45b7-92c0-9615b59c9afb\") " pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"
Sep 10 17:20:31.837065 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.836884    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69b6f0c6-12c7-45b7-92c0-9615b59c9afb-cert\") pod \"lws-controller-manager-8655464fff-lvz59\" (UID: \"69b6f0c6-12c7-45b7-92c0-9615b59c9afb\") " pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"
Sep 10 17:20:31.837065 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.836901    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/69b6f0c6-12c7-45b7-92c0-9615b59c9afb-metrics-cert\") pod \"lws-controller-manager-8655464fff-lvz59\" (UID: \"69b6f0c6-12c7-45b7-92c0-9615b59c9afb\") " pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"
Sep 10 17:20:31.837065 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.836924    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/69b6f0c6-12c7-45b7-92c0-9615b59c9afb-manager-config\") pod \"lws-controller-manager-8655464fff-lvz59\" (UID: \"69b6f0c6-12c7-45b7-92c0-9615b59c9afb\") " pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"
Sep 10 17:20:31.837575 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.837553    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/69b6f0c6-12c7-45b7-92c0-9615b59c9afb-manager-config\") pod \"lws-controller-manager-8655464fff-lvz59\" (UID: \"69b6f0c6-12c7-45b7-92c0-9615b59c9afb\") " pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"
Sep 10 17:20:31.839392 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.839369    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69b6f0c6-12c7-45b7-92c0-9615b59c9afb-cert\") pod \"lws-controller-manager-8655464fff-lvz59\" (UID: \"69b6f0c6-12c7-45b7-92c0-9615b59c9afb\") " pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"
Sep 10 17:20:31.839537 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.839517    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/69b6f0c6-12c7-45b7-92c0-9615b59c9afb-metrics-cert\") pod \"lws-controller-manager-8655464fff-lvz59\" (UID: \"69b6f0c6-12c7-45b7-92c0-9615b59c9afb\") " pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"
Sep 10 17:20:31.860916 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.860892    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6ccf\" (UniqueName: \"kubernetes.io/projected/69b6f0c6-12c7-45b7-92c0-9615b59c9afb-kube-api-access-s6ccf\") pod \"lws-controller-manager-8655464fff-lvz59\" (UID: \"69b6f0c6-12c7-45b7-92c0-9615b59c9afb\") " pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"
Sep 10 17:20:31.953950 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:31.953881    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"
Sep 10 17:20:32.293342 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:32.293268    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:32.296375 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:32.296354    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"]
Sep 10 17:20:32.297771 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:20:32.297744    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69b6f0c6_12c7_45b7_92c0_9615b59c9afb.slice/crio-c898f1fc33602b57a52942bc224cf4e89191fa4030f429ad1d3125b4864428d3 WatchSource:0}: Error finding container c898f1fc33602b57a52942bc224cf4e89191fa4030f429ad1d3125b4864428d3: Status 404 returned error can't find the container with id c898f1fc33602b57a52942bc224cf4e89191fa4030f429ad1d3125b4864428d3
Sep 10 17:20:32.687898 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:32.687822    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:32.903970 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:32.903935    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59" event={"ID":"69b6f0c6-12c7-45b7-92c0-9615b59c9afb","Type":"ContainerStarted","Data":"c898f1fc33602b57a52942bc224cf4e89191fa4030f429ad1d3125b4864428d3"}
Sep 10 17:20:33.029388 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:33.029349    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:33.572254 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:33.572226    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:39.415143 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:39.415032    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:39.605415 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:39.605382    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:42.184542 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:42.184502    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:43.221429 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:43.221387    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7bbsn"]
Sep 10 17:20:43.225231 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:43.225213    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7bbsn"
Sep 10 17:20:43.235605 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:43.235582    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 10 17:20:43.236583 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:43.236564    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-jdfmf\""
Sep 10 17:20:43.247510 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:43.247481    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7bbsn"]
Sep 10 17:20:43.298363 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:43.298328    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:43.345208 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:43.345172    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/745988e5-d7ff-43e3-bf69-3e7372106216-cert\") pod \"odh-model-controller-7998c67dc4-7bbsn\" (UID: \"745988e5-d7ff-43e3-bf69-3e7372106216\") " pod="opendatahub/odh-model-controller-7998c67dc4-7bbsn"
Sep 10 17:20:43.345401 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:43.345322    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plsrx\" (UniqueName: \"kubernetes.io/projected/745988e5-d7ff-43e3-bf69-3e7372106216-kube-api-access-plsrx\") pod \"odh-model-controller-7998c67dc4-7bbsn\" (UID: \"745988e5-d7ff-43e3-bf69-3e7372106216\") " pod="opendatahub/odh-model-controller-7998c67dc4-7bbsn"
Sep 10 17:20:43.446112 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:43.446079    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plsrx\" (UniqueName: \"kubernetes.io/projected/745988e5-d7ff-43e3-bf69-3e7372106216-kube-api-access-plsrx\") pod \"odh-model-controller-7998c67dc4-7bbsn\" (UID: \"745988e5-d7ff-43e3-bf69-3e7372106216\") " pod="opendatahub/odh-model-controller-7998c67dc4-7bbsn"
Sep 10 17:20:43.446329 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:43.446170    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/745988e5-d7ff-43e3-bf69-3e7372106216-cert\") pod \"odh-model-controller-7998c67dc4-7bbsn\" (UID: \"745988e5-d7ff-43e3-bf69-3e7372106216\") " pod="opendatahub/odh-model-controller-7998c67dc4-7bbsn"
Sep 10 17:20:43.448618 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:43.448592    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/745988e5-d7ff-43e3-bf69-3e7372106216-cert\") pod \"odh-model-controller-7998c67dc4-7bbsn\" (UID: \"745988e5-d7ff-43e3-bf69-3e7372106216\") " pod="opendatahub/odh-model-controller-7998c67dc4-7bbsn"
Sep 10 17:20:43.463719 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:43.463690    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plsrx\" (UniqueName: \"kubernetes.io/projected/745988e5-d7ff-43e3-bf69-3e7372106216-kube-api-access-plsrx\") pod \"odh-model-controller-7998c67dc4-7bbsn\" (UID: \"745988e5-d7ff-43e3-bf69-3e7372106216\") " pod="opendatahub/odh-model-controller-7998c67dc4-7bbsn"
Sep 10 17:20:43.481819 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:43.481748    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:43.539139 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:43.539087    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7bbsn"
Sep 10 17:20:43.645101 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:43.645064    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:43.838936 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:43.838905    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7bbsn"]
Sep 10 17:20:43.841962 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:20:43.841934    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod745988e5_d7ff_43e3_bf69_3e7372106216.slice/crio-d30a7dda8e5e7ef83221436ee7bb1eb1ca61912577b9c2827a502503aa4e3213 WatchSource:0}: Error finding container d30a7dda8e5e7ef83221436ee7bb1eb1ca61912577b9c2827a502503aa4e3213: Status 404 returned error can't find the container with id d30a7dda8e5e7ef83221436ee7bb1eb1ca61912577b9c2827a502503aa4e3213
Sep 10 17:20:43.870668 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:43.870588    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:43.960427 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:43.960381    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7bbsn" event={"ID":"745988e5-d7ff-43e3-bf69-3e7372106216","Type":"ContainerStarted","Data":"d30a7dda8e5e7ef83221436ee7bb1eb1ca61912577b9c2827a502503aa4e3213"}
Sep 10 17:20:44.065207 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.065093    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:44.196197 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.196095    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"]
Sep 10 17:20:44.201017 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.200978    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.204966 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.204938    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 17:20:44.205109 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.204937    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 17:20:44.205891 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.205870    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 17:20:44.205891 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.205883    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-clfrz\""
Sep 10 17:20:44.250097 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.250015    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"]
Sep 10 17:20:44.315467 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.315429    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:44.355228 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.355184    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4daf4486-5c21-4743-ba60-484fbfc18dd9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.355406 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.355244    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4daf4486-5c21-4743-ba60-484fbfc18dd9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.355406 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.355326    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4daf4486-5c21-4743-ba60-484fbfc18dd9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.355406 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.355382    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4daf4486-5c21-4743-ba60-484fbfc18dd9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.355575 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.355414    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4daf4486-5c21-4743-ba60-484fbfc18dd9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.355575 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.355439    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4daf4486-5c21-4743-ba60-484fbfc18dd9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.355575 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.355528    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86lv9\" (UniqueName: \"kubernetes.io/projected/4daf4486-5c21-4743-ba60-484fbfc18dd9-kube-api-access-86lv9\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.355688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.355573    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4daf4486-5c21-4743-ba60-484fbfc18dd9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.355688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.355665    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4daf4486-5c21-4743-ba60-484fbfc18dd9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.456411 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.456373    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4daf4486-5c21-4743-ba60-484fbfc18dd9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.456534 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.456436    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4daf4486-5c21-4743-ba60-484fbfc18dd9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.456534 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.456462    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4daf4486-5c21-4743-ba60-484fbfc18dd9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.456655 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.456580    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86lv9\" (UniqueName: \"kubernetes.io/projected/4daf4486-5c21-4743-ba60-484fbfc18dd9-kube-api-access-86lv9\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.456655 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.456627    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4daf4486-5c21-4743-ba60-484fbfc18dd9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.456762 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.456715    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4daf4486-5c21-4743-ba60-484fbfc18dd9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.456762 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.456753    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4daf4486-5c21-4743-ba60-484fbfc18dd9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.456871 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.456785    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4daf4486-5c21-4743-ba60-484fbfc18dd9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.456871 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.456841    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4daf4486-5c21-4743-ba60-484fbfc18dd9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.456979 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.456898    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4daf4486-5c21-4743-ba60-484fbfc18dd9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.457110 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.457088    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4daf4486-5c21-4743-ba60-484fbfc18dd9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.457225 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.456846    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4daf4486-5c21-4743-ba60-484fbfc18dd9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.457577 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.457527    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4daf4486-5c21-4743-ba60-484fbfc18dd9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.457674 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.457581    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4daf4486-5c21-4743-ba60-484fbfc18dd9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.459620 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.459565    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4daf4486-5c21-4743-ba60-484fbfc18dd9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.460233 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.460202    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4daf4486-5c21-4743-ba60-484fbfc18dd9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.483434 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.483403    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4daf4486-5c21-4743-ba60-484fbfc18dd9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.483590 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.483557    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86lv9\" (UniqueName: \"kubernetes.io/projected/4daf4486-5c21-4743-ba60-484fbfc18dd9-kube-api-access-86lv9\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz\" (UID: \"4daf4486-5c21-4743-ba60-484fbfc18dd9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.507897 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.507870    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:44.515789 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.515760    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:44.528585 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.528558    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c7l6n"]
Sep 10 17:20:44.533903 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.533882    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-c7l6n"
Sep 10 17:20:44.539368 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.539344    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-4h6dp\""
Sep 10 17:20:44.539571 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.539557    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 10 17:20:44.559385 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.558614    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c7l6n"]
Sep 10 17:20:44.660203 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.660168    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c6b4d0c-ca19-4f6e-a766-05e618791c31-tls-certs\") pod \"model-serving-api-54d89bd5fc-c7l6n\" (UID: \"0c6b4d0c-ca19-4f6e-a766-05e618791c31\") " pod="opendatahub/model-serving-api-54d89bd5fc-c7l6n"
Sep 10 17:20:44.660397 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.660222    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhcjm\" (UniqueName: \"kubernetes.io/projected/0c6b4d0c-ca19-4f6e-a766-05e618791c31-kube-api-access-zhcjm\") pod \"model-serving-api-54d89bd5fc-c7l6n\" (UID: \"0c6b4d0c-ca19-4f6e-a766-05e618791c31\") " pod="opendatahub/model-serving-api-54d89bd5fc-c7l6n"
Sep 10 17:20:44.688806 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.688757    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:44.762166 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.761677    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c6b4d0c-ca19-4f6e-a766-05e618791c31-tls-certs\") pod \"model-serving-api-54d89bd5fc-c7l6n\" (UID: \"0c6b4d0c-ca19-4f6e-a766-05e618791c31\") " pod="opendatahub/model-serving-api-54d89bd5fc-c7l6n"
Sep 10 17:20:44.762166 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.761755    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhcjm\" (UniqueName: \"kubernetes.io/projected/0c6b4d0c-ca19-4f6e-a766-05e618791c31-kube-api-access-zhcjm\") pod \"model-serving-api-54d89bd5fc-c7l6n\" (UID: \"0c6b4d0c-ca19-4f6e-a766-05e618791c31\") " pod="opendatahub/model-serving-api-54d89bd5fc-c7l6n"
Sep 10 17:20:44.765767 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.765718    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c6b4d0c-ca19-4f6e-a766-05e618791c31-tls-certs\") pod \"model-serving-api-54d89bd5fc-c7l6n\" (UID: \"0c6b4d0c-ca19-4f6e-a766-05e618791c31\") " pod="opendatahub/model-serving-api-54d89bd5fc-c7l6n"
Sep 10 17:20:44.777476 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.777450    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhcjm\" (UniqueName: \"kubernetes.io/projected/0c6b4d0c-ca19-4f6e-a766-05e618791c31-kube-api-access-zhcjm\") pod \"model-serving-api-54d89bd5fc-c7l6n\" (UID: \"0c6b4d0c-ca19-4f6e-a766-05e618791c31\") " pod="opendatahub/model-serving-api-54d89bd5fc-c7l6n"
Sep 10 17:20:44.826620 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.826536    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:44.845836 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.845805    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"]
Sep 10 17:20:44.851607 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:20:44.851567    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4daf4486_5c21_4743_ba60_484fbfc18dd9.slice/crio-1ac3ad1007176f131a0e0d55d54d72791f1bbbb6fecd86d3f1e9e098e0ae11bc WatchSource:0}: Error finding container 1ac3ad1007176f131a0e0d55d54d72791f1bbbb6fecd86d3f1e9e098e0ae11bc: Status 404 returned error can't find the container with id 1ac3ad1007176f131a0e0d55d54d72791f1bbbb6fecd86d3f1e9e098e0ae11bc
Sep 10 17:20:44.852769 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.852748    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-c7l6n"
Sep 10 17:20:44.974786 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.974748    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59" event={"ID":"69b6f0c6-12c7-45b7-92c0-9615b59c9afb","Type":"ContainerStarted","Data":"f41b67e647463c77e18f5937ef884be7b05acc662d70aba3a93fec71c3a11ce8"}
Sep 10 17:20:44.974969 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.974941    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"
Sep 10 17:20:44.976832 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:44.976806    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz" event={"ID":"4daf4486-5c21-4743-ba60-484fbfc18dd9","Type":"ContainerStarted","Data":"1ac3ad1007176f131a0e0d55d54d72791f1bbbb6fecd86d3f1e9e098e0ae11bc"}
Sep 10 17:20:45.021532 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:45.021418    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59" podStartSLOduration=1.884731921 podStartE2EDuration="14.021395356s" totalImagesPullingTime="12.136663435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:20:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:20:45.018639048 +0000 UTC m=+448.699812017" watchObservedRunningTime="2026-09-10 17:20:45.021395356 +0000 UTC m=+448.702568318"
Sep 10 17:20:45.208219 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:45.208176    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c7l6n"]
Sep 10 17:20:45.208979 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:20:45.208953    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c6b4d0c_ca19_4f6e_a766_05e618791c31.slice/crio-a944e437cc50814b3b77fa1ad455c915356ed89925f4a7698973f762fcd3d415 WatchSource:0}: Error finding container a944e437cc50814b3b77fa1ad455c915356ed89925f4a7698973f762fcd3d415: Status 404 returned error can't find the container with id a944e437cc50814b3b77fa1ad455c915356ed89925f4a7698973f762fcd3d415
Sep 10 17:20:45.985154 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:45.985103    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-c7l6n" event={"ID":"0c6b4d0c-ca19-4f6e-a766-05e618791c31","Type":"ContainerStarted","Data":"a944e437cc50814b3b77fa1ad455c915356ed89925f4a7698973f762fcd3d415"}
Sep 10 17:20:46.969504 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:46.969378    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:46.997092 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:46.996548    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7bbsn" event={"ID":"745988e5-d7ff-43e3-bf69-3e7372106216","Type":"ContainerStarted","Data":"213b8e97cd0b4ba7ec61b3671fd2809218686fcf497e8ac9990a434a4a3e9ef3"}
Sep 10 17:20:46.997092 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:46.996992    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-7bbsn"
Sep 10 17:20:47.035947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:47.034343    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-7bbsn" podStartSLOduration=1.035331148 podStartE2EDuration="4.034324295s" totalImagesPullingTime="2.998993147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:20:47.030428393 +0000 UTC m=+450.711601338" watchObservedRunningTime="2026-09-10 17:20:47.034324295 +0000 UTC m=+450.715497238"
Sep 10 17:20:49.033061 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:49.033015    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:49.163953 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:49.163913    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:50.622638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:50.622605    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:20:50.622899 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:50.622670    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:20:50.622899 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:50.622701    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:20:51.028802 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.028759    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz" event={"ID":"4daf4486-5c21-4743-ba60-484fbfc18dd9","Type":"ContainerStarted","Data":"587724f2b82ccacfae3d9230ab8fb5dec8f346c64de5eb5a1d9a2016f977d8f3"}
Sep 10 17:20:51.030924 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.030888    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-c7l6n" event={"ID":"0c6b4d0c-ca19-4f6e-a766-05e618791c31","Type":"ContainerStarted","Data":"9f2fd09bbb5308de3046f2729b7c2499624b536033da76890dd9db8cc51f2011"}
Sep 10 17:20:51.031092 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.030999    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-c7l6n"
Sep 10 17:20:51.111544 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.111494    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz" podStartSLOduration=1.345066737 podStartE2EDuration="7.11148031s" totalImagesPullingTime="5.766413573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:20:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:20:51.106958405 +0000 UTC m=+454.788131352" watchObservedRunningTime="2026-09-10 17:20:51.11148031 +0000 UTC m=+454.792653253"
Sep 10 17:20:51.124204 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.121289    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6589f5969b-7gkhx"]
Sep 10 17:20:51.126478 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.126450    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.144691 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.144659    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6589f5969b-7gkhx"]
Sep 10 17:20:51.145674 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.145627    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-c7l6n" podStartSLOduration=1.737879853 podStartE2EDuration="7.145613903s" totalImagesPullingTime="5.40773405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:20:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:20:51.141392507 +0000 UTC m=+454.822565473" watchObservedRunningTime="2026-09-10 17:20:51.145613903 +0000 UTC m=+454.826786847"
Sep 10 17:20:51.226181 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.226152    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3db60130-0267-461f-9083-928fc93a1d4b-trusted-ca-bundle\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.226339 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.226204    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3db60130-0267-461f-9083-928fc93a1d4b-console-serving-cert\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.226339 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.226235    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3db60130-0267-461f-9083-928fc93a1d4b-console-oauth-config\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.226339 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.226251    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3db60130-0267-461f-9083-928fc93a1d4b-service-ca\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.226460 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.226422    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3db60130-0267-461f-9083-928fc93a1d4b-console-config\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.226521 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.226504    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2nmf\" (UniqueName: \"kubernetes.io/projected/3db60130-0267-461f-9083-928fc93a1d4b-kube-api-access-w2nmf\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.226607 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.226589    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3db60130-0267-461f-9083-928fc93a1d4b-oauth-serving-cert\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.311117 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.311027    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:51.328183 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.328148    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3db60130-0267-461f-9083-928fc93a1d4b-console-oauth-config\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.328325 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.328193    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3db60130-0267-461f-9083-928fc93a1d4b-service-ca\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.328325 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.328241    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3db60130-0267-461f-9083-928fc93a1d4b-console-config\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.328451 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.328388    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2nmf\" (UniqueName: \"kubernetes.io/projected/3db60130-0267-461f-9083-928fc93a1d4b-kube-api-access-w2nmf\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.328503 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.328452    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3db60130-0267-461f-9083-928fc93a1d4b-oauth-serving-cert\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.328616 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.328535    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3db60130-0267-461f-9083-928fc93a1d4b-trusted-ca-bundle\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.328616 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.328579    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3db60130-0267-461f-9083-928fc93a1d4b-console-serving-cert\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.329078 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.329042    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3db60130-0267-461f-9083-928fc93a1d4b-service-ca\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.329234 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.329081    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3db60130-0267-461f-9083-928fc93a1d4b-console-config\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.329234 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.329213    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3db60130-0267-461f-9083-928fc93a1d4b-oauth-serving-cert\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.329433 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.329412    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3db60130-0267-461f-9083-928fc93a1d4b-trusted-ca-bundle\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.330643 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.330622    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3db60130-0267-461f-9083-928fc93a1d4b-console-oauth-config\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.330990 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.330972    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3db60130-0267-461f-9083-928fc93a1d4b-console-serving-cert\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.341805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.341780    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2nmf\" (UniqueName: \"kubernetes.io/projected/3db60130-0267-461f-9083-928fc93a1d4b-kube-api-access-w2nmf\") pod \"console-6589f5969b-7gkhx\" (UID: \"3db60130-0267-461f-9083-928fc93a1d4b\") " pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.435397 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.435361    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:20:51.466489 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.466454    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:20:51.516491 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.516464    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:51.521439 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.521416    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:51.750220 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:51.750194    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6589f5969b-7gkhx"]
Sep 10 17:20:51.752330 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:20:51.752301    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3db60130_0267_461f_9083_928fc93a1d4b.slice/crio-a1e5bf42fad27a06e536ca946d26d3c80b974530994442c5e617d31610be5aae WatchSource:0}: Error finding container a1e5bf42fad27a06e536ca946d26d3c80b974530994442c5e617d31610be5aae: Status 404 returned error can't find the container with id a1e5bf42fad27a06e536ca946d26d3c80b974530994442c5e617d31610be5aae
Sep 10 17:20:52.037315 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:52.037225    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6589f5969b-7gkhx" event={"ID":"3db60130-0267-461f-9083-928fc93a1d4b","Type":"ContainerStarted","Data":"5825f29825f5decfed7eac178710a3ea17dbe845113c1c8df25ccc194f11b7b3"}
Sep 10 17:20:52.037315 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:52.037282    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6589f5969b-7gkhx" event={"ID":"3db60130-0267-461f-9083-928fc93a1d4b","Type":"ContainerStarted","Data":"a1e5bf42fad27a06e536ca946d26d3c80b974530994442c5e617d31610be5aae"}
Sep 10 17:20:52.037680 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:52.037659    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:52.038693 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:52.038671    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz"
Sep 10 17:20:52.106322 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:52.106270    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6589f5969b-7gkhx" podStartSLOduration=1.106254005 podStartE2EDuration="1.106254005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:20:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:20:52.104351044 +0000 UTC m=+455.785523987" watchObservedRunningTime="2026-09-10 17:20:52.106254005 +0000 UTC m=+455.787426948"
Sep 10 17:20:55.987342 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:55.987310    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8655464fff-lvz59"
Sep 10 17:20:58.005576 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:58.005547    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-7bbsn"
Sep 10 17:20:59.751589 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:20:59.751553    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:21:00.218333 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:00.218198    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:21:00.604854 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:00.604821    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:21:00.873472 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:00.873390    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:21:01.436332 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:01.436299    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:21:01.436509 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:01.436383    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:21:01.441894 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:01.441873    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:21:02.043546 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:02.043517    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-c7l6n"
Sep 10 17:21:02.099078 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:02.099049    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6589f5969b-7gkhx"
Sep 10 17:21:02.223413 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:02.223383    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85dcfddb4d-cvqqs"]
Sep 10 17:21:02.532095 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:02.532065    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:21:02.712050 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:02.712018    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:21:27.242739 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.242696    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85dcfddb4d-cvqqs" podUID="c64a49b1-c790-4f19-8703-21b6a7157e3d" containerName="console" containerID="cri-o://a2bf800f98374f07bd1535c15fb2df30301cfcc562012cb881c6f18f93be0d6f" gracePeriod=15
Sep 10 17:21:27.490437 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.490414    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:21:27.646852 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.646777    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-service-ca\") pod \"c64a49b1-c790-4f19-8703-21b6a7157e3d\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") "
Sep 10 17:21:27.646852 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.646823    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dm2sl\" (UniqueName: \"kubernetes.io/projected/c64a49b1-c790-4f19-8703-21b6a7157e3d-kube-api-access-dm2sl\") pod \"c64a49b1-c790-4f19-8703-21b6a7157e3d\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") "
Sep 10 17:21:27.646852 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.646845    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c64a49b1-c790-4f19-8703-21b6a7157e3d-console-serving-cert\") pod \"c64a49b1-c790-4f19-8703-21b6a7157e3d\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") "
Sep 10 17:21:27.647106 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.646894    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-trusted-ca-bundle\") pod \"c64a49b1-c790-4f19-8703-21b6a7157e3d\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") "
Sep 10 17:21:27.647106 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.646927    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-oauth-serving-cert\") pod \"c64a49b1-c790-4f19-8703-21b6a7157e3d\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") "
Sep 10 17:21:27.647106 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.646970    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c64a49b1-c790-4f19-8703-21b6a7157e3d-console-oauth-config\") pod \"c64a49b1-c790-4f19-8703-21b6a7157e3d\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") "
Sep 10 17:21:27.647106 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.647003    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-console-config\") pod \"c64a49b1-c790-4f19-8703-21b6a7157e3d\" (UID: \"c64a49b1-c790-4f19-8703-21b6a7157e3d\") "
Sep 10 17:21:27.647343 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.647222    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-service-ca" (OuterVolumeSpecName: "service-ca") pod "c64a49b1-c790-4f19-8703-21b6a7157e3d" (UID: "c64a49b1-c790-4f19-8703-21b6a7157e3d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:21:27.647544 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.647511    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c64a49b1-c790-4f19-8703-21b6a7157e3d" (UID: "c64a49b1-c790-4f19-8703-21b6a7157e3d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:21:27.647639 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.647542    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c64a49b1-c790-4f19-8703-21b6a7157e3d" (UID: "c64a49b1-c790-4f19-8703-21b6a7157e3d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:21:27.647639 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.647605    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-console-config" (OuterVolumeSpecName: "console-config") pod "c64a49b1-c790-4f19-8703-21b6a7157e3d" (UID: "c64a49b1-c790-4f19-8703-21b6a7157e3d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:21:27.649181 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.649152    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c64a49b1-c790-4f19-8703-21b6a7157e3d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c64a49b1-c790-4f19-8703-21b6a7157e3d" (UID: "c64a49b1-c790-4f19-8703-21b6a7157e3d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:21:27.649472 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.649454    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c64a49b1-c790-4f19-8703-21b6a7157e3d-kube-api-access-dm2sl" (OuterVolumeSpecName: "kube-api-access-dm2sl") pod "c64a49b1-c790-4f19-8703-21b6a7157e3d" (UID: "c64a49b1-c790-4f19-8703-21b6a7157e3d"). InnerVolumeSpecName "kube-api-access-dm2sl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:21:27.649518 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.649494    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c64a49b1-c790-4f19-8703-21b6a7157e3d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c64a49b1-c790-4f19-8703-21b6a7157e3d" (UID: "c64a49b1-c790-4f19-8703-21b6a7157e3d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:21:27.747986 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.747959    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-trusted-ca-bundle\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:21:27.747986 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.747984    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-oauth-serving-cert\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:21:27.748198 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.747997    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c64a49b1-c790-4f19-8703-21b6a7157e3d-console-oauth-config\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:21:27.748198 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.748009    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-console-config\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:21:27.748198 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.748021    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c64a49b1-c790-4f19-8703-21b6a7157e3d-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:21:27.748198 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.748035    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dm2sl\" (UniqueName: \"kubernetes.io/projected/c64a49b1-c790-4f19-8703-21b6a7157e3d-kube-api-access-dm2sl\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:21:27.748198 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:27.748047    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c64a49b1-c790-4f19-8703-21b6a7157e3d-console-serving-cert\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:21:28.228065 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:28.228028    2566 generic.go:356] "Generic (PLEG): container finished" podID="c64a49b1-c790-4f19-8703-21b6a7157e3d" containerID="a2bf800f98374f07bd1535c15fb2df30301cfcc562012cb881c6f18f93be0d6f" exitCode=0
Sep 10 17:21:28.228311 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:28.228072    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85dcfddb4d-cvqqs"
Sep 10 17:21:28.228311 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:28.228100    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85dcfddb4d-cvqqs" event={"ID":"c64a49b1-c790-4f19-8703-21b6a7157e3d","Type":"ContainerDied","Data":"a2bf800f98374f07bd1535c15fb2df30301cfcc562012cb881c6f18f93be0d6f"}
Sep 10 17:21:28.228311 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:28.228160    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85dcfddb4d-cvqqs" event={"ID":"c64a49b1-c790-4f19-8703-21b6a7157e3d","Type":"ContainerDied","Data":"11254c5cfb0cbadee084f989cc74c577e3879dcff0b40073c4afe45252cf7a3b"}
Sep 10 17:21:28.228311 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:28.228182    2566 scope.go:117] "RemoveContainer" containerID="a2bf800f98374f07bd1535c15fb2df30301cfcc562012cb881c6f18f93be0d6f"
Sep 10 17:21:28.249528 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:28.249368    2566 scope.go:117] "RemoveContainer" containerID="a2bf800f98374f07bd1535c15fb2df30301cfcc562012cb881c6f18f93be0d6f"
Sep 10 17:21:28.249755 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:21:28.249657    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2bf800f98374f07bd1535c15fb2df30301cfcc562012cb881c6f18f93be0d6f\": container with ID starting with a2bf800f98374f07bd1535c15fb2df30301cfcc562012cb881c6f18f93be0d6f not found: ID does not exist" containerID="a2bf800f98374f07bd1535c15fb2df30301cfcc562012cb881c6f18f93be0d6f"
Sep 10 17:21:28.249755 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:28.249679    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2bf800f98374f07bd1535c15fb2df30301cfcc562012cb881c6f18f93be0d6f"} err="failed to get container status \"a2bf800f98374f07bd1535c15fb2df30301cfcc562012cb881c6f18f93be0d6f\": rpc error: code = NotFound desc = could not find container \"a2bf800f98374f07bd1535c15fb2df30301cfcc562012cb881c6f18f93be0d6f\": container with ID starting with a2bf800f98374f07bd1535c15fb2df30301cfcc562012cb881c6f18f93be0d6f not found: ID does not exist"
Sep 10 17:21:28.260214 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:21:28.260187    2566 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc64a49b1_c790_4f19_8703_21b6a7157e3d.slice/crio-11254c5cfb0cbadee084f989cc74c577e3879dcff0b40073c4afe45252cf7a3b\": RecentStats: unable to find data in memory cache]"
Sep 10 17:21:28.261207 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:28.261186    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85dcfddb4d-cvqqs"]
Sep 10 17:21:28.263256 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:28.263236    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85dcfddb4d-cvqqs"]
Sep 10 17:21:28.833725 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:28.833690    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c64a49b1-c790-4f19-8703-21b6a7157e3d" path="/var/lib/kubelet/pods/c64a49b1-c790-4f19-8703-21b6a7157e3d/volumes"
Sep 10 17:21:44.497841 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:44.497801    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:21:44.769383 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:44.769296    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:21:53.035020 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.034990    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb"]
Sep 10 17:21:53.035393 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.035376    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c64a49b1-c790-4f19-8703-21b6a7157e3d" containerName="console"
Sep 10 17:21:53.035393 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.035389    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c64a49b1-c790-4f19-8703-21b6a7157e3d" containerName="console"
Sep 10 17:21:53.035473 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.035467    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c64a49b1-c790-4f19-8703-21b6a7157e3d" containerName="console"
Sep 10 17:21:53.038939 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.038921    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb"
Sep 10 17:21:53.043473 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.043454    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 17:21:53.043578 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.043534    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9gr7t\""
Sep 10 17:21:53.043883 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.043866    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 17:21:53.065696 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.065671    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb"]
Sep 10 17:21:53.067829 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.067808    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a3bcb36-f332-4079-80bd-9b6dff08fb5e-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb\" (UID: \"8a3bcb36-f332-4079-80bd-9b6dff08fb5e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb"
Sep 10 17:21:53.067932 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.067875    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a3bcb36-f332-4079-80bd-9b6dff08fb5e-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb\" (UID: \"8a3bcb36-f332-4079-80bd-9b6dff08fb5e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb"
Sep 10 17:21:53.067932 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.067919    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9fcq\" (UniqueName: \"kubernetes.io/projected/8a3bcb36-f332-4079-80bd-9b6dff08fb5e-kube-api-access-f9fcq\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb\" (UID: \"8a3bcb36-f332-4079-80bd-9b6dff08fb5e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb"
Sep 10 17:21:53.169452 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.169412    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a3bcb36-f332-4079-80bd-9b6dff08fb5e-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb\" (UID: \"8a3bcb36-f332-4079-80bd-9b6dff08fb5e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb"
Sep 10 17:21:53.169624 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.169489    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a3bcb36-f332-4079-80bd-9b6dff08fb5e-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb\" (UID: \"8a3bcb36-f332-4079-80bd-9b6dff08fb5e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb"
Sep 10 17:21:53.169624 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.169532    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9fcq\" (UniqueName: \"kubernetes.io/projected/8a3bcb36-f332-4079-80bd-9b6dff08fb5e-kube-api-access-f9fcq\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb\" (UID: \"8a3bcb36-f332-4079-80bd-9b6dff08fb5e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb"
Sep 10 17:21:53.169880 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.169862    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a3bcb36-f332-4079-80bd-9b6dff08fb5e-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb\" (UID: \"8a3bcb36-f332-4079-80bd-9b6dff08fb5e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb"
Sep 10 17:21:53.169921 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.169869    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a3bcb36-f332-4079-80bd-9b6dff08fb5e-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb\" (UID: \"8a3bcb36-f332-4079-80bd-9b6dff08fb5e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb"
Sep 10 17:21:53.174941 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.174917    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7"]
Sep 10 17:21:53.178706 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.178689    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7"
Sep 10 17:21:53.193788 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.193762    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9fcq\" (UniqueName: \"kubernetes.io/projected/8a3bcb36-f332-4079-80bd-9b6dff08fb5e-kube-api-access-f9fcq\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb\" (UID: \"8a3bcb36-f332-4079-80bd-9b6dff08fb5e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb"
Sep 10 17:21:53.205157 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.205135    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7"]
Sep 10 17:21:53.270380 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.270354    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mprc8\" (UniqueName: \"kubernetes.io/projected/263c33bc-e02a-4b0c-a3f5-aea15e7da17d-kube-api-access-mprc8\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7\" (UID: \"263c33bc-e02a-4b0c-a3f5-aea15e7da17d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7"
Sep 10 17:21:53.270511 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.270409    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/263c33bc-e02a-4b0c-a3f5-aea15e7da17d-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7\" (UID: \"263c33bc-e02a-4b0c-a3f5-aea15e7da17d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7"
Sep 10 17:21:53.270550 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.270515    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/263c33bc-e02a-4b0c-a3f5-aea15e7da17d-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7\" (UID: \"263c33bc-e02a-4b0c-a3f5-aea15e7da17d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7"
Sep 10 17:21:53.305776 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.305707    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx"]
Sep 10 17:21:53.309463 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.309439    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx"
Sep 10 17:21:53.328030 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.328007    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx"]
Sep 10 17:21:53.348083 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.348061    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb"
Sep 10 17:21:53.371654 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.371624    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx\" (UID: \"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx"
Sep 10 17:21:53.371779 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.371688    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/263c33bc-e02a-4b0c-a3f5-aea15e7da17d-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7\" (UID: \"263c33bc-e02a-4b0c-a3f5-aea15e7da17d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7"
Sep 10 17:21:53.371779 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.371735    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mprc8\" (UniqueName: \"kubernetes.io/projected/263c33bc-e02a-4b0c-a3f5-aea15e7da17d-kube-api-access-mprc8\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7\" (UID: \"263c33bc-e02a-4b0c-a3f5-aea15e7da17d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7"
Sep 10 17:21:53.371779 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.371774    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd2tc\" (UniqueName: \"kubernetes.io/projected/8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a-kube-api-access-qd2tc\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx\" (UID: \"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx"
Sep 10 17:21:53.371928 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.371798    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx\" (UID: \"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx"
Sep 10 17:21:53.371928 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.371841    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/263c33bc-e02a-4b0c-a3f5-aea15e7da17d-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7\" (UID: \"263c33bc-e02a-4b0c-a3f5-aea15e7da17d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7"
Sep 10 17:21:53.372231 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.372207    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/263c33bc-e02a-4b0c-a3f5-aea15e7da17d-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7\" (UID: \"263c33bc-e02a-4b0c-a3f5-aea15e7da17d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7"
Sep 10 17:21:53.372231 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.372225    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/263c33bc-e02a-4b0c-a3f5-aea15e7da17d-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7\" (UID: \"263c33bc-e02a-4b0c-a3f5-aea15e7da17d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7"
Sep 10 17:21:53.383138 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.383101    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mprc8\" (UniqueName: \"kubernetes.io/projected/263c33bc-e02a-4b0c-a3f5-aea15e7da17d-kube-api-access-mprc8\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7\" (UID: \"263c33bc-e02a-4b0c-a3f5-aea15e7da17d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7"
Sep 10 17:21:53.442283 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.442255    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx"]
Sep 10 17:21:53.447040 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.447025    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx"
Sep 10 17:21:53.473305 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.473282    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx"]
Sep 10 17:21:53.473415 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.473323    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx\" (UID: \"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx"
Sep 10 17:21:53.473415 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.473367    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tshn7\" (UniqueName: \"kubernetes.io/projected/8aeea350-47f2-4179-ad62-eaef2312d6ba-kube-api-access-tshn7\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx\" (UID: \"8aeea350-47f2-4179-ad62-eaef2312d6ba\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx"
Sep 10 17:21:53.473415 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.473411    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8aeea350-47f2-4179-ad62-eaef2312d6ba-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx\" (UID: \"8aeea350-47f2-4179-ad62-eaef2312d6ba\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx"
Sep 10 17:21:53.473589 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.473430    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd2tc\" (UniqueName: \"kubernetes.io/projected/8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a-kube-api-access-qd2tc\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx\" (UID: \"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx"
Sep 10 17:21:53.473589 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.473479    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx\" (UID: \"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx"
Sep 10 17:21:53.473589 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.473521    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8aeea350-47f2-4179-ad62-eaef2312d6ba-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx\" (UID: \"8aeea350-47f2-4179-ad62-eaef2312d6ba\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx"
Sep 10 17:21:53.473751 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.473719    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx\" (UID: \"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx"
Sep 10 17:21:53.473793 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.473764    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx\" (UID: \"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx"
Sep 10 17:21:53.488229 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.488204    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7"
Sep 10 17:21:53.500202 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.500178    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd2tc\" (UniqueName: \"kubernetes.io/projected/8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a-kube-api-access-qd2tc\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx\" (UID: \"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx"
Sep 10 17:21:53.574887 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.574799    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8aeea350-47f2-4179-ad62-eaef2312d6ba-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx\" (UID: \"8aeea350-47f2-4179-ad62-eaef2312d6ba\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx"
Sep 10 17:21:53.574887 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.574856    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8aeea350-47f2-4179-ad62-eaef2312d6ba-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx\" (UID: \"8aeea350-47f2-4179-ad62-eaef2312d6ba\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx"
Sep 10 17:21:53.575114 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.574923    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tshn7\" (UniqueName: \"kubernetes.io/projected/8aeea350-47f2-4179-ad62-eaef2312d6ba-kube-api-access-tshn7\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx\" (UID: \"8aeea350-47f2-4179-ad62-eaef2312d6ba\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx"
Sep 10 17:21:53.575323 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.575293    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8aeea350-47f2-4179-ad62-eaef2312d6ba-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx\" (UID: \"8aeea350-47f2-4179-ad62-eaef2312d6ba\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx"
Sep 10 17:21:53.575395 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.575347    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8aeea350-47f2-4179-ad62-eaef2312d6ba-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx\" (UID: \"8aeea350-47f2-4179-ad62-eaef2312d6ba\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx"
Sep 10 17:21:53.594907 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.594876    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tshn7\" (UniqueName: \"kubernetes.io/projected/8aeea350-47f2-4179-ad62-eaef2312d6ba-kube-api-access-tshn7\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx\" (UID: \"8aeea350-47f2-4179-ad62-eaef2312d6ba\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx"
Sep 10 17:21:53.618783 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.618758    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx"
Sep 10 17:21:53.655216 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.655192    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb"]
Sep 10 17:21:53.658132 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:21:53.658083    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a3bcb36_f332_4079_80bd_9b6dff08fb5e.slice/crio-05add3c1173b11bb4a0d725961f1199b985a087c9bc3ca4f92ba242ad595a220 WatchSource:0}: Error finding container 05add3c1173b11bb4a0d725961f1199b985a087c9bc3ca4f92ba242ad595a220: Status 404 returned error can't find the container with id 05add3c1173b11bb4a0d725961f1199b985a087c9bc3ca4f92ba242ad595a220
Sep 10 17:21:53.756920 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.756892    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx"
Sep 10 17:21:53.792252 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.792216    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7"]
Sep 10 17:21:53.919290 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:53.919259    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx"]
Sep 10 17:21:53.951892 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:21:53.951860    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b841c98_bf5d_4a6d_8ca3_d1d6edb8c53a.slice/crio-c8d14927232f962a409c1ba0d594f39c2dba046ed7e7f2b1aa624b397713de22 WatchSource:0}: Error finding container c8d14927232f962a409c1ba0d594f39c2dba046ed7e7f2b1aa624b397713de22: Status 404 returned error can't find the container with id c8d14927232f962a409c1ba0d594f39c2dba046ed7e7f2b1aa624b397713de22
Sep 10 17:21:54.121742 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:54.121718    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx"]
Sep 10 17:21:54.127074 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:21:54.127047    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aeea350_47f2_4179_ad62_eaef2312d6ba.slice/crio-e2796bfc4e8b7f0204963ebb0f5c2522b69918a7e0347fab02c4db950c52dbac WatchSource:0}: Error finding container e2796bfc4e8b7f0204963ebb0f5c2522b69918a7e0347fab02c4db950c52dbac: Status 404 returned error can't find the container with id e2796bfc4e8b7f0204963ebb0f5c2522b69918a7e0347fab02c4db950c52dbac
Sep 10 17:21:54.359771 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:54.359737    2566 generic.go:356] "Generic (PLEG): container finished" podID="263c33bc-e02a-4b0c-a3f5-aea15e7da17d" containerID="fe6ebd77d735ff7e06d823b8f113dcf5c2047a00bdc09702c9bbe505abb19f38" exitCode=0
Sep 10 17:21:54.359976 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:54.359830    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7" event={"ID":"263c33bc-e02a-4b0c-a3f5-aea15e7da17d","Type":"ContainerDied","Data":"fe6ebd77d735ff7e06d823b8f113dcf5c2047a00bdc09702c9bbe505abb19f38"}
Sep 10 17:21:54.359976 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:54.359891    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7" event={"ID":"263c33bc-e02a-4b0c-a3f5-aea15e7da17d","Type":"ContainerStarted","Data":"6d081e695c19863ab6818309e293284ac7ccac56561e1a8f26aabcbd4475f745"}
Sep 10 17:21:54.361509 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:54.361482    2566 generic.go:356] "Generic (PLEG): container finished" podID="8a3bcb36-f332-4079-80bd-9b6dff08fb5e" containerID="b37a305f318c0bbe56c851e70daf1a29dad679024bf4c2e572a310c04fb20684" exitCode=0
Sep 10 17:21:54.361634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:54.361563    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb" event={"ID":"8a3bcb36-f332-4079-80bd-9b6dff08fb5e","Type":"ContainerDied","Data":"b37a305f318c0bbe56c851e70daf1a29dad679024bf4c2e572a310c04fb20684"}
Sep 10 17:21:54.361634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:54.361612    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb" event={"ID":"8a3bcb36-f332-4079-80bd-9b6dff08fb5e","Type":"ContainerStarted","Data":"05add3c1173b11bb4a0d725961f1199b985a087c9bc3ca4f92ba242ad595a220"}
Sep 10 17:21:54.363275 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:54.363245    2566 generic.go:356] "Generic (PLEG): container finished" podID="8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a" containerID="1b719bc0f4ade073418f9a4ca74f27ba879d935857ba29345b64f5b6f49a7fe9" exitCode=0
Sep 10 17:21:54.363370 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:54.363297    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx" event={"ID":"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a","Type":"ContainerDied","Data":"1b719bc0f4ade073418f9a4ca74f27ba879d935857ba29345b64f5b6f49a7fe9"}
Sep 10 17:21:54.363370 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:54.363336    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx" event={"ID":"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a","Type":"ContainerStarted","Data":"c8d14927232f962a409c1ba0d594f39c2dba046ed7e7f2b1aa624b397713de22"}
Sep 10 17:21:54.364999 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:54.364978    2566 generic.go:356] "Generic (PLEG): container finished" podID="8aeea350-47f2-4179-ad62-eaef2312d6ba" containerID="e4713fd86539856d0488b7e2bf24cf25c8d08d9246eaed7f0f830e0dfe988725" exitCode=0
Sep 10 17:21:54.365110 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:54.365029    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx" event={"ID":"8aeea350-47f2-4179-ad62-eaef2312d6ba","Type":"ContainerDied","Data":"e4713fd86539856d0488b7e2bf24cf25c8d08d9246eaed7f0f830e0dfe988725"}
Sep 10 17:21:54.365110 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:54.365073    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx" event={"ID":"8aeea350-47f2-4179-ad62-eaef2312d6ba","Type":"ContainerStarted","Data":"e2796bfc4e8b7f0204963ebb0f5c2522b69918a7e0347fab02c4db950c52dbac"}
Sep 10 17:21:55.372347 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:55.372320    2566 generic.go:356] "Generic (PLEG): container finished" podID="8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a" containerID="3b60d35c0240acbc4881bced71e721d553541de6727bb6289bd3a084f8b1219f" exitCode=0
Sep 10 17:21:55.372701 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:55.372408    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx" event={"ID":"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a","Type":"ContainerDied","Data":"3b60d35c0240acbc4881bced71e721d553541de6727bb6289bd3a084f8b1219f"}
Sep 10 17:21:55.374475 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:55.374453    2566 generic.go:356] "Generic (PLEG): container finished" podID="263c33bc-e02a-4b0c-a3f5-aea15e7da17d" containerID="e17570d2171d35a0aa8e48d6912c5e25f14519ea83a018fd663bb94f3c17a658" exitCode=0
Sep 10 17:21:55.374579 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:55.374504    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7" event={"ID":"263c33bc-e02a-4b0c-a3f5-aea15e7da17d","Type":"ContainerDied","Data":"e17570d2171d35a0aa8e48d6912c5e25f14519ea83a018fd663bb94f3c17a658"}
Sep 10 17:21:56.382577 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:56.382543    2566 generic.go:356] "Generic (PLEG): container finished" podID="8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a" containerID="90945254c6ef46fad4b2892c22c7721925d89e086956e64aac07f49b5b53955a" exitCode=0
Sep 10 17:21:56.382965 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:56.382619    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx" event={"ID":"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a","Type":"ContainerDied","Data":"90945254c6ef46fad4b2892c22c7721925d89e086956e64aac07f49b5b53955a"}
Sep 10 17:21:56.384602 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:56.384581    2566 generic.go:356] "Generic (PLEG): container finished" podID="8aeea350-47f2-4179-ad62-eaef2312d6ba" containerID="fd2f760c1e6967d33ceb2a84adcfa195ef8758db30094f798937aca3664f375d" exitCode=0
Sep 10 17:21:56.384737 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:56.384669    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx" event={"ID":"8aeea350-47f2-4179-ad62-eaef2312d6ba","Type":"ContainerDied","Data":"fd2f760c1e6967d33ceb2a84adcfa195ef8758db30094f798937aca3664f375d"}
Sep 10 17:21:56.386952 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:56.386931    2566 generic.go:356] "Generic (PLEG): container finished" podID="263c33bc-e02a-4b0c-a3f5-aea15e7da17d" containerID="ce705784594ca7ab191c5dc12a22a277ea22d45d1133fe375ca76aa56900ee2e" exitCode=0
Sep 10 17:21:56.387065 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:56.386981    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7" event={"ID":"263c33bc-e02a-4b0c-a3f5-aea15e7da17d","Type":"ContainerDied","Data":"ce705784594ca7ab191c5dc12a22a277ea22d45d1133fe375ca76aa56900ee2e"}
Sep 10 17:21:57.394293 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.394260    2566 generic.go:356] "Generic (PLEG): container finished" podID="8aeea350-47f2-4179-ad62-eaef2312d6ba" containerID="c4b0cd73c27db0315dd7bb2d7c26335ecac2f3a624693b9cf7a71a568dc50dae" exitCode=0
Sep 10 17:21:57.394694 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.394325    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx" event={"ID":"8aeea350-47f2-4179-ad62-eaef2312d6ba","Type":"ContainerDied","Data":"c4b0cd73c27db0315dd7bb2d7c26335ecac2f3a624693b9cf7a71a568dc50dae"}
Sep 10 17:21:57.525002 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.524975    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7"
Sep 10 17:21:57.554151 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.554104    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx"
Sep 10 17:21:57.610119 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.610093    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qd2tc\" (UniqueName: \"kubernetes.io/projected/8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a-kube-api-access-qd2tc\") pod \"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a\" (UID: \"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a\") "
Sep 10 17:21:57.610291 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.610278    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a-util\") pod \"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a\" (UID: \"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a\") "
Sep 10 17:21:57.610337 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.610304    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/263c33bc-e02a-4b0c-a3f5-aea15e7da17d-util\") pod \"263c33bc-e02a-4b0c-a3f5-aea15e7da17d\" (UID: \"263c33bc-e02a-4b0c-a3f5-aea15e7da17d\") "
Sep 10 17:21:57.610337 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.610319    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/263c33bc-e02a-4b0c-a3f5-aea15e7da17d-bundle\") pod \"263c33bc-e02a-4b0c-a3f5-aea15e7da17d\" (UID: \"263c33bc-e02a-4b0c-a3f5-aea15e7da17d\") "
Sep 10 17:21:57.610435 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.610339    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mprc8\" (UniqueName: \"kubernetes.io/projected/263c33bc-e02a-4b0c-a3f5-aea15e7da17d-kube-api-access-mprc8\") pod \"263c33bc-e02a-4b0c-a3f5-aea15e7da17d\" (UID: \"263c33bc-e02a-4b0c-a3f5-aea15e7da17d\") "
Sep 10 17:21:57.610435 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.610371    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a-bundle\") pod \"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a\" (UID: \"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a\") "
Sep 10 17:21:57.610934 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.610892    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a-bundle" (OuterVolumeSpecName: "bundle") pod "8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a" (UID: "8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:21:57.611109 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.611066    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/263c33bc-e02a-4b0c-a3f5-aea15e7da17d-bundle" (OuterVolumeSpecName: "bundle") pod "263c33bc-e02a-4b0c-a3f5-aea15e7da17d" (UID: "263c33bc-e02a-4b0c-a3f5-aea15e7da17d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:21:57.612602 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.612559    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a-kube-api-access-qd2tc" (OuterVolumeSpecName: "kube-api-access-qd2tc") pod "8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a" (UID: "8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a"). InnerVolumeSpecName "kube-api-access-qd2tc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:21:57.612745 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.612673    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/263c33bc-e02a-4b0c-a3f5-aea15e7da17d-kube-api-access-mprc8" (OuterVolumeSpecName: "kube-api-access-mprc8") pod "263c33bc-e02a-4b0c-a3f5-aea15e7da17d" (UID: "263c33bc-e02a-4b0c-a3f5-aea15e7da17d"). InnerVolumeSpecName "kube-api-access-mprc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:21:57.616443 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.616421    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a-util" (OuterVolumeSpecName: "util") pod "8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a" (UID: "8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:21:57.617628 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.617609    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/263c33bc-e02a-4b0c-a3f5-aea15e7da17d-util" (OuterVolumeSpecName: "util") pod "263c33bc-e02a-4b0c-a3f5-aea15e7da17d" (UID: "263c33bc-e02a-4b0c-a3f5-aea15e7da17d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:21:57.711283 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.711213    2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a-util\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:21:57.711283 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.711240    2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/263c33bc-e02a-4b0c-a3f5-aea15e7da17d-util\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:21:57.711283 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.711249    2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/263c33bc-e02a-4b0c-a3f5-aea15e7da17d-bundle\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:21:57.711283 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.711260    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mprc8\" (UniqueName: \"kubernetes.io/projected/263c33bc-e02a-4b0c-a3f5-aea15e7da17d-kube-api-access-mprc8\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:21:57.711283 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.711270    2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a-bundle\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:21:57.711283 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:57.711279    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd2tc\" (UniqueName: \"kubernetes.io/projected/8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a-kube-api-access-qd2tc\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:21:58.401830 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:58.401795    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7"
Sep 10 17:21:58.402421 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:58.401797    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvs4h7" event={"ID":"263c33bc-e02a-4b0c-a3f5-aea15e7da17d","Type":"ContainerDied","Data":"6d081e695c19863ab6818309e293284ac7ccac56561e1a8f26aabcbd4475f745"}
Sep 10 17:21:58.402421 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:58.401926    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d081e695c19863ab6818309e293284ac7ccac56561e1a8f26aabcbd4475f745"
Sep 10 17:21:58.403791 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:58.403762    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx"
Sep 10 17:21:58.403791 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:58.403782    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62dllx" event={"ID":"8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a","Type":"ContainerDied","Data":"c8d14927232f962a409c1ba0d594f39c2dba046ed7e7f2b1aa624b397713de22"}
Sep 10 17:21:58.403984 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:58.403815    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8d14927232f962a409c1ba0d594f39c2dba046ed7e7f2b1aa624b397713de22"
Sep 10 17:21:58.532492 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:58.532468    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx"
Sep 10 17:21:58.619221 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:58.619196    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tshn7\" (UniqueName: \"kubernetes.io/projected/8aeea350-47f2-4179-ad62-eaef2312d6ba-kube-api-access-tshn7\") pod \"8aeea350-47f2-4179-ad62-eaef2312d6ba\" (UID: \"8aeea350-47f2-4179-ad62-eaef2312d6ba\") "
Sep 10 17:21:58.619377 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:58.619258    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8aeea350-47f2-4179-ad62-eaef2312d6ba-util\") pod \"8aeea350-47f2-4179-ad62-eaef2312d6ba\" (UID: \"8aeea350-47f2-4179-ad62-eaef2312d6ba\") "
Sep 10 17:21:58.619377 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:58.619293    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8aeea350-47f2-4179-ad62-eaef2312d6ba-bundle\") pod \"8aeea350-47f2-4179-ad62-eaef2312d6ba\" (UID: \"8aeea350-47f2-4179-ad62-eaef2312d6ba\") "
Sep 10 17:21:58.619806 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:58.619752    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aeea350-47f2-4179-ad62-eaef2312d6ba-bundle" (OuterVolumeSpecName: "bundle") pod "8aeea350-47f2-4179-ad62-eaef2312d6ba" (UID: "8aeea350-47f2-4179-ad62-eaef2312d6ba"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:21:58.621443 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:58.621414    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8aeea350-47f2-4179-ad62-eaef2312d6ba-kube-api-access-tshn7" (OuterVolumeSpecName: "kube-api-access-tshn7") pod "8aeea350-47f2-4179-ad62-eaef2312d6ba" (UID: "8aeea350-47f2-4179-ad62-eaef2312d6ba"). InnerVolumeSpecName "kube-api-access-tshn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:21:58.624556 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:58.624537    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aeea350-47f2-4179-ad62-eaef2312d6ba-util" (OuterVolumeSpecName: "util") pod "8aeea350-47f2-4179-ad62-eaef2312d6ba" (UID: "8aeea350-47f2-4179-ad62-eaef2312d6ba"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:21:58.720717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:58.720642    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tshn7\" (UniqueName: \"kubernetes.io/projected/8aeea350-47f2-4179-ad62-eaef2312d6ba-kube-api-access-tshn7\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:21:58.720717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:58.720670    2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8aeea350-47f2-4179-ad62-eaef2312d6ba-util\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:21:58.720717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:58.720683    2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8aeea350-47f2-4179-ad62-eaef2312d6ba-bundle\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:21:59.411328 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:59.411301    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx"
Sep 10 17:21:59.411328 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:59.411314    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7w26hx" event={"ID":"8aeea350-47f2-4179-ad62-eaef2312d6ba","Type":"ContainerDied","Data":"e2796bfc4e8b7f0204963ebb0f5c2522b69918a7e0347fab02c4db950c52dbac"}
Sep 10 17:21:59.411731 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:21:59.411354    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2796bfc4e8b7f0204963ebb0f5c2522b69918a7e0347fab02c4db950c52dbac"
Sep 10 17:22:08.461113 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:08.461068    2566 generic.go:356] "Generic (PLEG): container finished" podID="8a3bcb36-f332-4079-80bd-9b6dff08fb5e" containerID="c2e62dd319544f2d0fd2f4c1949905ccdc9e53d3f8bbceac6814777dc97ea546" exitCode=0
Sep 10 17:22:08.461573 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:08.461158    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb" event={"ID":"8a3bcb36-f332-4079-80bd-9b6dff08fb5e","Type":"ContainerDied","Data":"c2e62dd319544f2d0fd2f4c1949905ccdc9e53d3f8bbceac6814777dc97ea546"}
Sep 10 17:22:09.469386 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:09.469306    2566 generic.go:356] "Generic (PLEG): container finished" podID="8a3bcb36-f332-4079-80bd-9b6dff08fb5e" containerID="8808dbf5a68fd992fde449181a49c4a1c774cb420af5d3465a77bb325db780f4" exitCode=0
Sep 10 17:22:09.469386 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:09.469352    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb" event={"ID":"8a3bcb36-f332-4079-80bd-9b6dff08fb5e","Type":"ContainerDied","Data":"8808dbf5a68fd992fde449181a49c4a1c774cb420af5d3465a77bb325db780f4"}
Sep 10 17:22:10.601074 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:10.601047    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb"
Sep 10 17:22:10.722937 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:10.722904    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9fcq\" (UniqueName: \"kubernetes.io/projected/8a3bcb36-f332-4079-80bd-9b6dff08fb5e-kube-api-access-f9fcq\") pod \"8a3bcb36-f332-4079-80bd-9b6dff08fb5e\" (UID: \"8a3bcb36-f332-4079-80bd-9b6dff08fb5e\") "
Sep 10 17:22:10.723093 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:10.722946    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a3bcb36-f332-4079-80bd-9b6dff08fb5e-bundle\") pod \"8a3bcb36-f332-4079-80bd-9b6dff08fb5e\" (UID: \"8a3bcb36-f332-4079-80bd-9b6dff08fb5e\") "
Sep 10 17:22:10.723093 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:10.723002    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a3bcb36-f332-4079-80bd-9b6dff08fb5e-util\") pod \"8a3bcb36-f332-4079-80bd-9b6dff08fb5e\" (UID: \"8a3bcb36-f332-4079-80bd-9b6dff08fb5e\") "
Sep 10 17:22:10.723796 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:10.723760    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a3bcb36-f332-4079-80bd-9b6dff08fb5e-bundle" (OuterVolumeSpecName: "bundle") pod "8a3bcb36-f332-4079-80bd-9b6dff08fb5e" (UID: "8a3bcb36-f332-4079-80bd-9b6dff08fb5e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:22:10.724932 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:10.724909    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a3bcb36-f332-4079-80bd-9b6dff08fb5e-kube-api-access-f9fcq" (OuterVolumeSpecName: "kube-api-access-f9fcq") pod "8a3bcb36-f332-4079-80bd-9b6dff08fb5e" (UID: "8a3bcb36-f332-4079-80bd-9b6dff08fb5e"). InnerVolumeSpecName "kube-api-access-f9fcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:22:10.728185 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:10.728113    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a3bcb36-f332-4079-80bd-9b6dff08fb5e-util" (OuterVolumeSpecName: "util") pod "8a3bcb36-f332-4079-80bd-9b6dff08fb5e" (UID: "8a3bcb36-f332-4079-80bd-9b6dff08fb5e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:22:10.824485 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:10.824454    2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a3bcb36-f332-4079-80bd-9b6dff08fb5e-util\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:22:10.824485 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:10.824480    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9fcq\" (UniqueName: \"kubernetes.io/projected/8a3bcb36-f332-4079-80bd-9b6dff08fb5e-kube-api-access-f9fcq\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:22:10.824485 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:10.824490    2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a3bcb36-f332-4079-80bd-9b6dff08fb5e-bundle\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:22:11.482076 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:11.482049    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb"
Sep 10 17:22:11.482261 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:11.482049    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajd2cb" event={"ID":"8a3bcb36-f332-4079-80bd-9b6dff08fb5e","Type":"ContainerDied","Data":"05add3c1173b11bb4a0d725961f1199b985a087c9bc3ca4f92ba242ad595a220"}
Sep 10 17:22:11.482261 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:11.482183    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05add3c1173b11bb4a0d725961f1199b985a087c9bc3ca4f92ba242ad595a220"
Sep 10 17:22:15.745479 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:15.745445    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:16.543354 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:16.543315    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:19.148453 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148414    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-2xlwn"]
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148787    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="263c33bc-e02a-4b0c-a3f5-aea15e7da17d" containerName="util"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148800    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="263c33bc-e02a-4b0c-a3f5-aea15e7da17d" containerName="util"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148813    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a" containerName="pull"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148818    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a" containerName="pull"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148826    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a" containerName="util"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148831    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a" containerName="util"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148838    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aeea350-47f2-4179-ad62-eaef2312d6ba" containerName="pull"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148843    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aeea350-47f2-4179-ad62-eaef2312d6ba" containerName="pull"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148849    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aeea350-47f2-4179-ad62-eaef2312d6ba" containerName="util"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148853    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aeea350-47f2-4179-ad62-eaef2312d6ba" containerName="util"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148861    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a3bcb36-f332-4079-80bd-9b6dff08fb5e" containerName="util"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148866    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a3bcb36-f332-4079-80bd-9b6dff08fb5e" containerName="util"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148871    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="263c33bc-e02a-4b0c-a3f5-aea15e7da17d" containerName="extract"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148877    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="263c33bc-e02a-4b0c-a3f5-aea15e7da17d" containerName="extract"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148881    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a3bcb36-f332-4079-80bd-9b6dff08fb5e" containerName="pull"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148886    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a3bcb36-f332-4079-80bd-9b6dff08fb5e" containerName="pull"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148893    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a3bcb36-f332-4079-80bd-9b6dff08fb5e" containerName="extract"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148898    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a3bcb36-f332-4079-80bd-9b6dff08fb5e" containerName="extract"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148904    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a" containerName="extract"
Sep 10 17:22:19.148901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148909    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a" containerName="extract"
Sep 10 17:22:19.149627 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148918    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aeea350-47f2-4179-ad62-eaef2312d6ba" containerName="extract"
Sep 10 17:22:19.149627 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148925    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aeea350-47f2-4179-ad62-eaef2312d6ba" containerName="extract"
Sep 10 17:22:19.149627 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148932    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="263c33bc-e02a-4b0c-a3f5-aea15e7da17d" containerName="pull"
Sep 10 17:22:19.149627 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148937    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="263c33bc-e02a-4b0c-a3f5-aea15e7da17d" containerName="pull"
Sep 10 17:22:19.149627 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148993    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="263c33bc-e02a-4b0c-a3f5-aea15e7da17d" containerName="extract"
Sep 10 17:22:19.149627 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.148999    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b841c98-bf5d-4a6d-8ca3-d1d6edb8c53a" containerName="extract"
Sep 10 17:22:19.149627 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.149005    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a3bcb36-f332-4079-80bd-9b6dff08fb5e" containerName="extract"
Sep 10 17:22:19.149627 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.149013    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aeea350-47f2-4179-ad62-eaef2312d6ba" containerName="extract"
Sep 10 17:22:19.153439 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.153416    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-2xlwn"
Sep 10 17:22:19.156408 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.156387    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 17:22:19.157262 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.157244    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-7jbzf\""
Sep 10 17:22:19.158853 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.158833    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 17:22:19.181593 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.181570    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-2xlwn"]
Sep 10 17:22:19.258752 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.258711    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:19.301887 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.301848    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwjmr\" (UniqueName: \"kubernetes.io/projected/65070453-dac7-4952-8ee1-be03f3968a8d-kube-api-access-vwjmr\") pod \"authorino-operator-76b478bd78-2xlwn\" (UID: \"65070453-dac7-4952-8ee1-be03f3968a8d\") " pod="kuadrant-system/authorino-operator-76b478bd78-2xlwn"
Sep 10 17:22:19.402773 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.402679    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwjmr\" (UniqueName: \"kubernetes.io/projected/65070453-dac7-4952-8ee1-be03f3968a8d-kube-api-access-vwjmr\") pod \"authorino-operator-76b478bd78-2xlwn\" (UID: \"65070453-dac7-4952-8ee1-be03f3968a8d\") " pod="kuadrant-system/authorino-operator-76b478bd78-2xlwn"
Sep 10 17:22:19.412400 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.412372    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwjmr\" (UniqueName: \"kubernetes.io/projected/65070453-dac7-4952-8ee1-be03f3968a8d-kube-api-access-vwjmr\") pod \"authorino-operator-76b478bd78-2xlwn\" (UID: \"65070453-dac7-4952-8ee1-be03f3968a8d\") " pod="kuadrant-system/authorino-operator-76b478bd78-2xlwn"
Sep 10 17:22:19.464341 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.464312    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-2xlwn"
Sep 10 17:22:19.747348 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.747325    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-2xlwn"]
Sep 10 17:22:19.749914 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:22:19.749892    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65070453_dac7_4952_8ee1_be03f3968a8d.slice/crio-9038a09582f9a73b42a1ac8c466fef305b757a947b5a8dfcffd739e3bbc3ed06 WatchSource:0}: Error finding container 9038a09582f9a73b42a1ac8c466fef305b757a947b5a8dfcffd739e3bbc3ed06: Status 404 returned error can't find the container with id 9038a09582f9a73b42a1ac8c466fef305b757a947b5a8dfcffd739e3bbc3ed06
Sep 10 17:22:19.966598 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:19.966562    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:20.536050 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:20.535999    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-2xlwn" event={"ID":"65070453-dac7-4952-8ee1-be03f3968a8d","Type":"ContainerStarted","Data":"9038a09582f9a73b42a1ac8c466fef305b757a947b5a8dfcffd739e3bbc3ed06"}
Sep 10 17:22:20.635968 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:20.635918    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:22.161519 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:22.161489    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:22.356351 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:22.356316    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:22.548965 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:22.548922    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-2xlwn" event={"ID":"65070453-dac7-4952-8ee1-be03f3968a8d","Type":"ContainerStarted","Data":"2e5b50e39d88effc65dd5e35ec3d12ed13f80a3cd5b031c01d44ad066d7dbce7"}
Sep 10 17:22:22.549209 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:22.549058    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-2xlwn"
Sep 10 17:22:22.579520 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:22.579471    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-2xlwn" podStartSLOduration=1.174768783 podStartE2EDuration="3.579456415s" totalImagesPullingTime="2.404687632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:22:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:22:22.5776269 +0000 UTC m=+546.258799857" watchObservedRunningTime="2026-09-10 17:22:22.579456415 +0000 UTC m=+546.260629358"
Sep 10 17:22:24.138117 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:24.138078    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:24.545858 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:24.545819    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:24.958562 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:24.958481    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:25.072833 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:25.072797    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:25.254065 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:25.254019    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:27.884560 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:27.884527    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"]
Sep 10 17:22:27.889421 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:27.889398    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"
Sep 10 17:22:27.892415 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:27.892393    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fpvh4\""
Sep 10 17:22:27.906291 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:27.906265    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"]
Sep 10 17:22:27.987284 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:27.987256    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8vhd\" (UniqueName: \"kubernetes.io/projected/9403d12c-dbc6-4cdd-9661-deeb6010da90-kube-api-access-q8vhd\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4ssts\" (UID: \"9403d12c-dbc6-4cdd-9661-deeb6010da90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"
Sep 10 17:22:27.987438 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:27.987316    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9403d12c-dbc6-4cdd-9661-deeb6010da90-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4ssts\" (UID: \"9403d12c-dbc6-4cdd-9661-deeb6010da90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"
Sep 10 17:22:28.088369 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:28.088326    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9403d12c-dbc6-4cdd-9661-deeb6010da90-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4ssts\" (UID: \"9403d12c-dbc6-4cdd-9661-deeb6010da90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"
Sep 10 17:22:28.088571 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:28.088416    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8vhd\" (UniqueName: \"kubernetes.io/projected/9403d12c-dbc6-4cdd-9661-deeb6010da90-kube-api-access-q8vhd\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4ssts\" (UID: \"9403d12c-dbc6-4cdd-9661-deeb6010da90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"
Sep 10 17:22:28.088823 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:28.088799    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9403d12c-dbc6-4cdd-9661-deeb6010da90-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4ssts\" (UID: \"9403d12c-dbc6-4cdd-9661-deeb6010da90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"
Sep 10 17:22:28.098877 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:28.098852    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8vhd\" (UniqueName: \"kubernetes.io/projected/9403d12c-dbc6-4cdd-9661-deeb6010da90-kube-api-access-q8vhd\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4ssts\" (UID: \"9403d12c-dbc6-4cdd-9661-deeb6010da90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"
Sep 10 17:22:28.199212 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:28.199121    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"
Sep 10 17:22:28.487692 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:28.487623    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"]
Sep 10 17:22:28.491623 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:22:28.491593    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9403d12c_dbc6_4cdd_9661_deeb6010da90.slice/crio-e29600b99d04a686849af723e73b5ecfceb0f31adf8b53ddaee18ab0b9a71e2d WatchSource:0}: Error finding container e29600b99d04a686849af723e73b5ecfceb0f31adf8b53ddaee18ab0b9a71e2d: Status 404 returned error can't find the container with id e29600b99d04a686849af723e73b5ecfceb0f31adf8b53ddaee18ab0b9a71e2d
Sep 10 17:22:28.582711 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:28.582682    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts" event={"ID":"9403d12c-dbc6-4cdd-9661-deeb6010da90","Type":"ContainerStarted","Data":"e29600b99d04a686849af723e73b5ecfceb0f31adf8b53ddaee18ab0b9a71e2d"}
Sep 10 17:22:29.540700 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:29.540663    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:29.666839 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:29.666757    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:29.961565 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:29.961476    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:30.071707 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:30.069080    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:32.988768 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:32.988693    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:33.087854 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:33.087811    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:33.556709 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:33.556678    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-2xlwn"
Sep 10 17:22:33.616312 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:33.616281    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts" event={"ID":"9403d12c-dbc6-4cdd-9661-deeb6010da90","Type":"ContainerStarted","Data":"bf5fc7654602c292b084af317b8b3390d5a1d56768f5377f65ea27b3c9850eb5"}
Sep 10 17:22:33.616495 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:33.616347    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"
Sep 10 17:22:33.647687 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:33.647643    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts" podStartSLOduration=2.593090174 podStartE2EDuration="6.647624673s" totalImagesPullingTime="4.054534499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:22:33.647487616 +0000 UTC m=+557.328660559" watchObservedRunningTime="2026-09-10 17:22:33.647624673 +0000 UTC m=+557.328797619"
Sep 10 17:22:35.420432 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:35.420383    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:35.728639 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:35.728546    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:36.800497 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:36.800464    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:36.920648 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:36.920613    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:38.369484 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:38.369451    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:38.516034 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:38.515999    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:42.223180 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:42.223116    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:42.345838 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:42.345808    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:42.483595 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:42.483508    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:44.486323 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:44.486288    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:44.624317 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:44.624287    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"
Sep 10 17:22:47.202415 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.202383    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"]
Sep 10 17:22:47.202899 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.202579    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts" podUID="9403d12c-dbc6-4cdd-9661-deeb6010da90" containerName="manager" containerID="cri-o://bf5fc7654602c292b084af317b8b3390d5a1d56768f5377f65ea27b3c9850eb5" gracePeriod=2
Sep 10 17:22:47.237756 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.237728    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"]
Sep 10 17:22:47.258032 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:22:47.257986    2566 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"9403d12c-dbc6-4cdd-9661-deeb6010da90\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-4ssts\": pods \"kuadrant-operator-controller-manager-76f8f686bf-4ssts\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"
Sep 10 17:22:47.388181 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.388148    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mz88z"]
Sep 10 17:22:47.388532 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.388519    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9403d12c-dbc6-4cdd-9661-deeb6010da90" containerName="manager"
Sep 10 17:22:47.388591 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.388533    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9403d12c-dbc6-4cdd-9661-deeb6010da90" containerName="manager"
Sep 10 17:22:47.388638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.388618    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9403d12c-dbc6-4cdd-9661-deeb6010da90" containerName="manager"
Sep 10 17:22:47.391925 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.391903    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mz88z"
Sep 10 17:22:47.396986 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:22:47.396957    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-4ssts\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" podUID="9403d12c-dbc6-4cdd-9661-deeb6010da90" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"
Sep 10 17:22:47.400635 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.400615    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-cmhc6\""
Sep 10 17:22:47.438468 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.438440    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mz88z"]
Sep 10 17:22:47.459887 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.459829    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqnqx\" (UniqueName: \"kubernetes.io/projected/a4d4cbb7-a40b-4ba5-934a-bb988edc9741-kube-api-access-xqnqx\") pod \"limitador-operator-controller-manager-74495c69f8-mz88z\" (UID: \"a4d4cbb7-a40b-4ba5-934a-bb988edc9741\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mz88z"
Sep 10 17:22:47.466866 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.466843    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"
Sep 10 17:22:47.503962 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:22:47.503927    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-4ssts\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" podUID="9403d12c-dbc6-4cdd-9661-deeb6010da90" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"
Sep 10 17:22:47.560513 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.560485    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8vhd\" (UniqueName: \"kubernetes.io/projected/9403d12c-dbc6-4cdd-9661-deeb6010da90-kube-api-access-q8vhd\") pod \"9403d12c-dbc6-4cdd-9661-deeb6010da90\" (UID: \"9403d12c-dbc6-4cdd-9661-deeb6010da90\") "
Sep 10 17:22:47.560654 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.560593    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9403d12c-dbc6-4cdd-9661-deeb6010da90-extensions-socket-volume\") pod \"9403d12c-dbc6-4cdd-9661-deeb6010da90\" (UID: \"9403d12c-dbc6-4cdd-9661-deeb6010da90\") "
Sep 10 17:22:47.560709 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.560695    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqnqx\" (UniqueName: \"kubernetes.io/projected/a4d4cbb7-a40b-4ba5-934a-bb988edc9741-kube-api-access-xqnqx\") pod \"limitador-operator-controller-manager-74495c69f8-mz88z\" (UID: \"a4d4cbb7-a40b-4ba5-934a-bb988edc9741\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mz88z"
Sep 10 17:22:47.561170 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.561136    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9403d12c-dbc6-4cdd-9661-deeb6010da90-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9403d12c-dbc6-4cdd-9661-deeb6010da90" (UID: "9403d12c-dbc6-4cdd-9661-deeb6010da90"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:22:47.562592 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.562568    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9403d12c-dbc6-4cdd-9661-deeb6010da90-kube-api-access-q8vhd" (OuterVolumeSpecName: "kube-api-access-q8vhd") pod "9403d12c-dbc6-4cdd-9661-deeb6010da90" (UID: "9403d12c-dbc6-4cdd-9661-deeb6010da90"). InnerVolumeSpecName "kube-api-access-q8vhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:22:47.584921 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.584895    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqnqx\" (UniqueName: \"kubernetes.io/projected/a4d4cbb7-a40b-4ba5-934a-bb988edc9741-kube-api-access-xqnqx\") pod \"limitador-operator-controller-manager-74495c69f8-mz88z\" (UID: \"a4d4cbb7-a40b-4ba5-934a-bb988edc9741\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mz88z"
Sep 10 17:22:47.661943 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.661918    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8vhd\" (UniqueName: \"kubernetes.io/projected/9403d12c-dbc6-4cdd-9661-deeb6010da90-kube-api-access-q8vhd\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:22:47.661943 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.661942    2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9403d12c-dbc6-4cdd-9661-deeb6010da90-extensions-socket-volume\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:22:47.696444 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.696413    2566 generic.go:356] "Generic (PLEG): container finished" podID="9403d12c-dbc6-4cdd-9661-deeb6010da90" containerID="bf5fc7654602c292b084af317b8b3390d5a1d56768f5377f65ea27b3c9850eb5" exitCode=0
Sep 10 17:22:47.696605 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.696472    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"
Sep 10 17:22:47.696605 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.696514    2566 scope.go:117] "RemoveContainer" containerID="bf5fc7654602c292b084af317b8b3390d5a1d56768f5377f65ea27b3c9850eb5"
Sep 10 17:22:47.699412 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:22:47.699384    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-4ssts\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" podUID="9403d12c-dbc6-4cdd-9661-deeb6010da90" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"
Sep 10 17:22:47.705539 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.705524    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mz88z"
Sep 10 17:22:47.706268 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.706252    2566 scope.go:117] "RemoveContainer" containerID="bf5fc7654602c292b084af317b8b3390d5a1d56768f5377f65ea27b3c9850eb5"
Sep 10 17:22:47.706551 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:22:47.706530    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf5fc7654602c292b084af317b8b3390d5a1d56768f5377f65ea27b3c9850eb5\": container with ID starting with bf5fc7654602c292b084af317b8b3390d5a1d56768f5377f65ea27b3c9850eb5 not found: ID does not exist" containerID="bf5fc7654602c292b084af317b8b3390d5a1d56768f5377f65ea27b3c9850eb5"
Sep 10 17:22:47.706626 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:47.706563    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf5fc7654602c292b084af317b8b3390d5a1d56768f5377f65ea27b3c9850eb5"} err="failed to get container status \"bf5fc7654602c292b084af317b8b3390d5a1d56768f5377f65ea27b3c9850eb5\": rpc error: code = NotFound desc = could not find container \"bf5fc7654602c292b084af317b8b3390d5a1d56768f5377f65ea27b3c9850eb5\": container with ID starting with bf5fc7654602c292b084af317b8b3390d5a1d56768f5377f65ea27b3c9850eb5 not found: ID does not exist"
Sep 10 17:22:47.710053 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:22:47.710008    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-4ssts\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" podUID="9403d12c-dbc6-4cdd-9661-deeb6010da90" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4ssts"
Sep 10 17:22:48.048537 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:48.048511    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mz88z"]
Sep 10 17:22:48.049676 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:22:48.049652    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4d4cbb7_a40b_4ba5_934a_bb988edc9741.slice/crio-9d4992d9aa4933caf992aeac13cef3fb37719a6a5bb23991b4091fca608fc1a9 WatchSource:0}: Error finding container 9d4992d9aa4933caf992aeac13cef3fb37719a6a5bb23991b4091fca608fc1a9: Status 404 returned error can't find the container with id 9d4992d9aa4933caf992aeac13cef3fb37719a6a5bb23991b4091fca608fc1a9
Sep 10 17:22:48.703686 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:48.703650    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mz88z" event={"ID":"a4d4cbb7-a40b-4ba5-934a-bb988edc9741","Type":"ContainerStarted","Data":"9d4992d9aa4933caf992aeac13cef3fb37719a6a5bb23991b4091fca608fc1a9"}
Sep 10 17:22:48.834346 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:48.834302    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9403d12c-dbc6-4cdd-9661-deeb6010da90" path="/var/lib/kubelet/pods/9403d12c-dbc6-4cdd-9661-deeb6010da90/volumes"
Sep 10 17:22:49.857561 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:49.857528    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:50.037471 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:50.037427    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:50.266975 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:50.266935    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:50.718953 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:50.718858    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mz88z" event={"ID":"a4d4cbb7-a40b-4ba5-934a-bb988edc9741","Type":"ContainerStarted","Data":"b7a7e95024f52f9c995b26c37fcf7d39a230ddd41da382baff9f17d17f760dc8"}
Sep 10 17:22:50.719148 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:50.718997    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mz88z"
Sep 10 17:22:50.761769 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:50.760286    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mz88z" podStartSLOduration=1.93394183 podStartE2EDuration="3.760264687s" totalImagesPullingTime="1.826322857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:22:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:22:50.755593744 +0000 UTC m=+574.436766688" watchObservedRunningTime="2026-09-10 17:22:50.760264687 +0000 UTC m=+574.441437632"
Sep 10 17:22:50.959188 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:50.959148    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2"]
Sep 10 17:22:50.963014 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:50.962991    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2"
Sep 10 17:22:50.969291 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:50.969231    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fpvh4\""
Sep 10 17:22:51.017396 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:51.017364    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2"]
Sep 10 17:22:51.096378 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:51.096350    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-748rs\" (UniqueName: \"kubernetes.io/projected/0c5d9784-5fa1-48a8-a857-d7bcc4756c3a-kube-api-access-748rs\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mnfx2\" (UID: \"0c5d9784-5fa1-48a8-a857-d7bcc4756c3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2"
Sep 10 17:22:51.096513 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:51.096399    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c5d9784-5fa1-48a8-a857-d7bcc4756c3a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mnfx2\" (UID: \"0c5d9784-5fa1-48a8-a857-d7bcc4756c3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2"
Sep 10 17:22:51.197001 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:51.196970    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c5d9784-5fa1-48a8-a857-d7bcc4756c3a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mnfx2\" (UID: \"0c5d9784-5fa1-48a8-a857-d7bcc4756c3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2"
Sep 10 17:22:51.197153 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:51.197074    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-748rs\" (UniqueName: \"kubernetes.io/projected/0c5d9784-5fa1-48a8-a857-d7bcc4756c3a-kube-api-access-748rs\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mnfx2\" (UID: \"0c5d9784-5fa1-48a8-a857-d7bcc4756c3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2"
Sep 10 17:22:51.197400 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:51.197384    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c5d9784-5fa1-48a8-a857-d7bcc4756c3a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mnfx2\" (UID: \"0c5d9784-5fa1-48a8-a857-d7bcc4756c3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2"
Sep 10 17:22:51.216307 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:51.216286    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-748rs\" (UniqueName: \"kubernetes.io/projected/0c5d9784-5fa1-48a8-a857-d7bcc4756c3a-kube-api-access-748rs\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mnfx2\" (UID: \"0c5d9784-5fa1-48a8-a857-d7bcc4756c3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2"
Sep 10 17:22:51.273380 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:51.273351    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2"
Sep 10 17:22:51.599100 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:51.599030    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2"]
Sep 10 17:22:51.602712 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:22:51.602683    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c5d9784_5fa1_48a8_a857_d7bcc4756c3a.slice/crio-b593201d16849071eec1b7c0fb580873831d5d5007d7613c659793f6cfda8e46 WatchSource:0}: Error finding container b593201d16849071eec1b7c0fb580873831d5d5007d7613c659793f6cfda8e46: Status 404 returned error can't find the container with id b593201d16849071eec1b7c0fb580873831d5d5007d7613c659793f6cfda8e46
Sep 10 17:22:51.728213 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:51.728177    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2" event={"ID":"0c5d9784-5fa1-48a8-a857-d7bcc4756c3a","Type":"ContainerStarted","Data":"386659ce699a98359c1f088771c6638c4f122cdddc56880aec4e94b895b17a8f"}
Sep 10 17:22:51.728386 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:51.728228    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2" event={"ID":"0c5d9784-5fa1-48a8-a857-d7bcc4756c3a","Type":"ContainerStarted","Data":"b593201d16849071eec1b7c0fb580873831d5d5007d7613c659793f6cfda8e46"}
Sep 10 17:22:51.728386 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:51.728312    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2"
Sep 10 17:22:51.795956 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:51.795910    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2" podStartSLOduration=1.795894732 podStartE2EDuration="1.795894732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:22:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:22:51.795031981 +0000 UTC m=+575.476204945" watchObservedRunningTime="2026-09-10 17:22:51.795894732 +0000 UTC m=+575.477067674"
Sep 10 17:22:53.169754 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:53.169713    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:53.552443 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:53.552408    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:57.085432 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:57.085397    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:22:57.197337 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:22:57.197295    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:23:01.730628 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:23:01.730588    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mz88z"
Sep 10 17:23:02.326783 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:23:02.326747    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:23:02.526108 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:23:02.526063    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:23:02.736844 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:23:02.736771    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2"
Sep 10 17:23:02.838389 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:23:02.838356    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:23:03.038701 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:23:03.038651    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:23:11.679921 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:23:11.679876    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:23:19.543389 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:23:19.543353    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:23:19.644526 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:23:19.644497    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:23:25.045728 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:23:25.045689    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:23:25.250321 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:23:25.250285    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:23:25.380373 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:23:25.380285    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:23:26.025654 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:23:26.025617    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:23:26.152691 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:23:26.152651    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:23:43.751998 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:23:43.751901    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:23:56.399872 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:23:56.399827    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:23:56.512209 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:23:56.512178    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:23:56.898410 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:23:56.898377    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:09.390474 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:09.390441    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:10.640464 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:10.640411    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:16.934236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:16.934204    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:17.082164 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:17.082109    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:17.213027 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:17.212946    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:17.335867 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:17.335833    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:20.563960 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:20.563925    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:20.668587 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:20.668558    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:26.742145 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:26.742080    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:26.981861 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:26.981812    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:27.506719 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:27.506683    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:32.638932 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:32.638895    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:37.075457 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:37.075420    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:47.209524 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:47.209488    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:50.231520 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:50.231481    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:52.618922 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:52.618888    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:52.781887 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:52.781848    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:54.126857 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:54.126819    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:54.260459 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:54.260414    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:54.504012 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:54.503983    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:54.861594 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:54.861514    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:55.181706 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:55.181625    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:24:55.351735 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:24:55.351697    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:25:07.449221 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:07.449174    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:25:07.587209 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:07.587178    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:25:13.940432 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:13.940348    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:25:35.071969 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:35.071932    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:25:48.023594 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:48.023546    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:25:53.611088 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.611057    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"]
Sep 10 17:25:53.614589 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.614572    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.619545 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.619524    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 17:25:53.619673 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.619631    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sb45l\""
Sep 10 17:25:53.620701 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.620679    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 17:25:53.620803 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.620716    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 10 17:25:53.631109 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.631080    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"]
Sep 10 17:25:53.651398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.651366    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:25:53.789069 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.789035    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bff45ed2-316b-4c3e-8578-62156fdcb4fb-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98jjj\" (UID: \"bff45ed2-316b-4c3e-8578-62156fdcb4fb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.789069 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.789070    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bff45ed2-316b-4c3e-8578-62156fdcb4fb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98jjj\" (UID: \"bff45ed2-316b-4c3e-8578-62156fdcb4fb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.789320 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.789111    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bff45ed2-316b-4c3e-8578-62156fdcb4fb-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98jjj\" (UID: \"bff45ed2-316b-4c3e-8578-62156fdcb4fb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.789320 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.789159    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bff45ed2-316b-4c3e-8578-62156fdcb4fb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98jjj\" (UID: \"bff45ed2-316b-4c3e-8578-62156fdcb4fb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.789320 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.789185    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz2zl\" (UniqueName: \"kubernetes.io/projected/bff45ed2-316b-4c3e-8578-62156fdcb4fb-kube-api-access-vz2zl\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98jjj\" (UID: \"bff45ed2-316b-4c3e-8578-62156fdcb4fb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.789320 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.789296    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bff45ed2-316b-4c3e-8578-62156fdcb4fb-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98jjj\" (UID: \"bff45ed2-316b-4c3e-8578-62156fdcb4fb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.890418 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.890337    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bff45ed2-316b-4c3e-8578-62156fdcb4fb-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98jjj\" (UID: \"bff45ed2-316b-4c3e-8578-62156fdcb4fb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.890418 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.890382    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bff45ed2-316b-4c3e-8578-62156fdcb4fb-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98jjj\" (UID: \"bff45ed2-316b-4c3e-8578-62156fdcb4fb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.890418 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.890406    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bff45ed2-316b-4c3e-8578-62156fdcb4fb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98jjj\" (UID: \"bff45ed2-316b-4c3e-8578-62156fdcb4fb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.890681 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.890436    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bff45ed2-316b-4c3e-8578-62156fdcb4fb-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98jjj\" (UID: \"bff45ed2-316b-4c3e-8578-62156fdcb4fb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.890681 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.890450    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bff45ed2-316b-4c3e-8578-62156fdcb4fb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98jjj\" (UID: \"bff45ed2-316b-4c3e-8578-62156fdcb4fb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.890681 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.890469    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz2zl\" (UniqueName: \"kubernetes.io/projected/bff45ed2-316b-4c3e-8578-62156fdcb4fb-kube-api-access-vz2zl\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98jjj\" (UID: \"bff45ed2-316b-4c3e-8578-62156fdcb4fb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.890874 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.890822    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bff45ed2-316b-4c3e-8578-62156fdcb4fb-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98jjj\" (UID: \"bff45ed2-316b-4c3e-8578-62156fdcb4fb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.890924 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.890869    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bff45ed2-316b-4c3e-8578-62156fdcb4fb-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98jjj\" (UID: \"bff45ed2-316b-4c3e-8578-62156fdcb4fb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.890962 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.890930    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bff45ed2-316b-4c3e-8578-62156fdcb4fb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98jjj\" (UID: \"bff45ed2-316b-4c3e-8578-62156fdcb4fb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.892595 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.892573    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bff45ed2-316b-4c3e-8578-62156fdcb4fb-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98jjj\" (UID: \"bff45ed2-316b-4c3e-8578-62156fdcb4fb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.892896 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.892877    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bff45ed2-316b-4c3e-8578-62156fdcb4fb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98jjj\" (UID: \"bff45ed2-316b-4c3e-8578-62156fdcb4fb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.900501 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.900479    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz2zl\" (UniqueName: \"kubernetes.io/projected/bff45ed2-316b-4c3e-8578-62156fdcb4fb-kube-api-access-vz2zl\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98jjj\" (UID: \"bff45ed2-316b-4c3e-8578-62156fdcb4fb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:53.925350 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:53.925323    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:25:54.201785 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:54.201756    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"]
Sep 10 17:25:54.202971 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:25:54.202947    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbff45ed2_316b_4c3e_8578_62156fdcb4fb.slice/crio-60060baf0ffcb32d97caa279286f0d419aa0d2b7eafb9193b594084dda450205 WatchSource:0}: Error finding container 60060baf0ffcb32d97caa279286f0d419aa0d2b7eafb9193b594084dda450205: Status 404 returned error can't find the container with id 60060baf0ffcb32d97caa279286f0d419aa0d2b7eafb9193b594084dda450205
Sep 10 17:25:54.204708 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:54.204691    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 17:25:54.752068 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:54.752032    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj" event={"ID":"bff45ed2-316b-4c3e-8578-62156fdcb4fb","Type":"ContainerStarted","Data":"60060baf0ffcb32d97caa279286f0d419aa0d2b7eafb9193b594084dda450205"}
Sep 10 17:25:55.759197 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:55.759160    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj" event={"ID":"bff45ed2-316b-4c3e-8578-62156fdcb4fb","Type":"ContainerStarted","Data":"8d255aded8ad9374d0e21ac71c1e57b627320ca761aa835568ec3425b35ebc0e"}
Sep 10 17:25:55.789568 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:25:55.789518    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj" podStartSLOduration=1.729866852 podStartE2EDuration="2.789503147s" totalImagesPullingTime="1.059636295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:25:55.78806218 +0000 UTC m=+759.469235168" watchObservedRunningTime="2026-09-10 17:25:55.789503147 +0000 UTC m=+759.470676089"
Sep 10 17:26:03.925625 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:26:03.925581    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:26:03.926013 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:26:03.925649    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:26:03.938225 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:26:03.938195    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:26:04.834054 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:26:04.834028    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98jjj"
Sep 10 17:26:07.680685 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:26:07.680651    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:26:13.469009 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:26:13.468973    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:26:22.569507 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:26:22.569464    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:27.695181 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:27.695140    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:27.791582 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:27.791543    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:28.839345 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:28.839311    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:28.935701 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:28.935673    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:44.147878 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:44.147836    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:44.277590 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:44.277558    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:44.557030 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:44.556992    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:44.678367 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:44.678335    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:45.630667 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:45.630614    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:45.774308 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:45.774274    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:46.091206 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:46.091170    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7926-48bb84"]
Sep 10 17:27:46.094704 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:46.094686    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7926-48bb84"
Sep 10 17:27:46.097267 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:46.097239    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4s82r\""
Sep 10 17:27:46.114784 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:46.114752    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7926-48bb84"]
Sep 10 17:27:46.124463 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:46.124433    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:46.158551 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:46.158524    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kwgj\" (UniqueName: \"kubernetes.io/projected/94d6e2ee-3117-4e25-98bc-27dd22e1b19f-kube-api-access-7kwgj\") pod \"test-curl-7926-48bb84\" (UID: \"94d6e2ee-3117-4e25-98bc-27dd22e1b19f\") " pod="opendatahub/test-curl-7926-48bb84"
Sep 10 17:27:46.259697 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:46.259663    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kwgj\" (UniqueName: \"kubernetes.io/projected/94d6e2ee-3117-4e25-98bc-27dd22e1b19f-kube-api-access-7kwgj\") pod \"test-curl-7926-48bb84\" (UID: \"94d6e2ee-3117-4e25-98bc-27dd22e1b19f\") " pod="opendatahub/test-curl-7926-48bb84"
Sep 10 17:27:46.269015 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:46.268982    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kwgj\" (UniqueName: \"kubernetes.io/projected/94d6e2ee-3117-4e25-98bc-27dd22e1b19f-kube-api-access-7kwgj\") pod \"test-curl-7926-48bb84\" (UID: \"94d6e2ee-3117-4e25-98bc-27dd22e1b19f\") " pod="opendatahub/test-curl-7926-48bb84"
Sep 10 17:27:46.404040 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:46.403968    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7926-48bb84"
Sep 10 17:27:46.697520 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:46.697442    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7926-48bb84"]
Sep 10 17:27:46.702139 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:27:46.702088    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94d6e2ee_3117_4e25_98bc_27dd22e1b19f.slice/crio-51fd419ab070ee6af388880aed086fa45575887b7004d13c9c3a78ef1e4926f3 WatchSource:0}: Error finding container 51fd419ab070ee6af388880aed086fa45575887b7004d13c9c3a78ef1e4926f3: Status 404 returned error can't find the container with id 51fd419ab070ee6af388880aed086fa45575887b7004d13c9c3a78ef1e4926f3
Sep 10 17:27:47.401922 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:47.401889    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7926-48bb84" event={"ID":"94d6e2ee-3117-4e25-98bc-27dd22e1b19f","Type":"ContainerStarted","Data":"9894cd8ed0af6810e3cfc58581036621bf115080c94504166ba12d90043a05fb"}
Sep 10 17:27:47.402069 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:47.401931    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7926-48bb84" event={"ID":"94d6e2ee-3117-4e25-98bc-27dd22e1b19f","Type":"ContainerStarted","Data":"51fd419ab070ee6af388880aed086fa45575887b7004d13c9c3a78ef1e4926f3"}
Sep 10 17:27:47.436176 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:47.436114    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7926-48bb84" podStartSLOduration=0.884195264 podStartE2EDuration="1.436098931s" totalImagesPullingTime="551.903667ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:27:47.434381628 +0000 UTC m=+871.115554572" watchObservedRunningTime="2026-09-10 17:27:47.436098931 +0000 UTC m=+871.117271900"
Sep 10 17:27:48.326677 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:48.326589    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:48.403973 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:48.403944    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7926-48bb84"]
Sep 10 17:27:48.414808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:48.414772    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7926-48bb84"]
Sep 10 17:27:48.561117 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:48.561086    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7926-7282da"]
Sep 10 17:27:48.561383 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:48.561328    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:48.561656 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:48.561637    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94d6e2ee-3117-4e25-98bc-27dd22e1b19f" containerName="test-curl-7926-48bb84"
Sep 10 17:27:48.561737 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:48.561658    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="94d6e2ee-3117-4e25-98bc-27dd22e1b19f" containerName="test-curl-7926-48bb84"
Sep 10 17:27:48.561737 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:48.561739    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="94d6e2ee-3117-4e25-98bc-27dd22e1b19f" containerName="test-curl-7926-48bb84"
Sep 10 17:27:48.564993 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:48.564974    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7926-7282da"
Sep 10 17:27:48.599098 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:48.599037    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7926-7282da"]
Sep 10 17:27:48.683515 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:48.683480    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q59d\" (UniqueName: \"kubernetes.io/projected/c2c7c124-0f47-444b-a819-b26dd7064d61-kube-api-access-9q59d\") pod \"test-curl-7926-7282da\" (UID: \"c2c7c124-0f47-444b-a819-b26dd7064d61\") " pod="opendatahub/test-curl-7926-7282da"
Sep 10 17:27:48.784496 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:48.784462    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q59d\" (UniqueName: \"kubernetes.io/projected/c2c7c124-0f47-444b-a819-b26dd7064d61-kube-api-access-9q59d\") pod \"test-curl-7926-7282da\" (UID: \"c2c7c124-0f47-444b-a819-b26dd7064d61\") " pod="opendatahub/test-curl-7926-7282da"
Sep 10 17:27:48.812716 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:48.812684    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q59d\" (UniqueName: \"kubernetes.io/projected/c2c7c124-0f47-444b-a819-b26dd7064d61-kube-api-access-9q59d\") pod \"test-curl-7926-7282da\" (UID: \"c2c7c124-0f47-444b-a819-b26dd7064d61\") " pod="opendatahub/test-curl-7926-7282da"
Sep 10 17:27:48.874823 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:48.874754    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7926-7282da"
Sep 10 17:27:49.154296 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:49.154271    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7926-7282da"]
Sep 10 17:27:49.155497 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:27:49.155468    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2c7c124_0f47_444b_a819_b26dd7064d61.slice/crio-d6bee95c7e3ec7e766a756a92b503a2668f35862c9d6e49eef67b355540617aa WatchSource:0}: Error finding container d6bee95c7e3ec7e766a756a92b503a2668f35862c9d6e49eef67b355540617aa: Status 404 returned error can't find the container with id d6bee95c7e3ec7e766a756a92b503a2668f35862c9d6e49eef67b355540617aa
Sep 10 17:27:49.415808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:49.415725    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7926-7282da" event={"ID":"c2c7c124-0f47-444b-a819-b26dd7064d61","Type":"ContainerStarted","Data":"9e9c2959aca312bab446813a5919de3cdd6d6804fff8e79e3744aa006adb5026"}
Sep 10 17:27:49.415808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:49.415769    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7926-7282da" event={"ID":"c2c7c124-0f47-444b-a819-b26dd7064d61","Type":"ContainerStarted","Data":"d6bee95c7e3ec7e766a756a92b503a2668f35862c9d6e49eef67b355540617aa"}
Sep 10 17:27:49.416284 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:49.415945    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7926-48bb84" podUID="94d6e2ee-3117-4e25-98bc-27dd22e1b19f" containerName="test-curl-7926-48bb84" containerID="cri-o://9894cd8ed0af6810e3cfc58581036621bf115080c94504166ba12d90043a05fb" gracePeriod=2
Sep 10 17:27:49.448852 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:49.448807    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7926-7282da" podStartSLOduration=1.448792189 podStartE2EDuration="1.448792189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:27:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:27:49.44623012 +0000 UTC m=+873.127403063" watchObservedRunningTime="2026-09-10 17:27:49.448792189 +0000 UTC m=+873.129965132"
Sep 10 17:27:49.675301 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:49.675227    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:49.810035 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:49.809996    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:49.977397 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:49.977368    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:50.217813 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.217736    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7926-7282da"]
Sep 10 17:27:50.224262 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.224237    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7926-7282da"]
Sep 10 17:27:50.358930 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.358887    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:50.422013 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:27:50.421987    2566 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7926-7282da" container="test-curl-7926-7282da"
Sep 10 17:27:50.476603 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.476517    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:50.590042 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.590015    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p"]
Sep 10 17:27:50.590508 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.590494    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2c7c124-0f47-444b-a819-b26dd7064d61" containerName="test-curl-7926-7282da"
Sep 10 17:27:50.590559 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.590510    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2c7c124-0f47-444b-a819-b26dd7064d61" containerName="test-curl-7926-7282da"
Sep 10 17:27:50.590601 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.590591    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2c7c124-0f47-444b-a819-b26dd7064d61" containerName="test-curl-7926-7282da"
Sep 10 17:27:50.593833 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.593819    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p"
Sep 10 17:27:50.598540 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.598508    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-8c46e8\""
Sep 10 17:27:50.598709 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.598694    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-8c46e8-dockercfg-4nqx9\""
Sep 10 17:27:50.611324 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.611299    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p"]
Sep 10 17:27:50.704310 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.704276    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23988339-54db-4c05-b849-f669e303ab29-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p\" (UID: \"23988339-54db-4c05-b849-f669e303ab29\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p"
Sep 10 17:27:50.704480 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.704351    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw4rn\" (UniqueName: \"kubernetes.io/projected/23988339-54db-4c05-b849-f669e303ab29-kube-api-access-bw4rn\") pod \"payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p\" (UID: \"23988339-54db-4c05-b849-f669e303ab29\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p"
Sep 10 17:27:50.752424 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.752390    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:50.805493 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.805464    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23988339-54db-4c05-b849-f669e303ab29-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p\" (UID: \"23988339-54db-4c05-b849-f669e303ab29\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p"
Sep 10 17:27:50.805634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.805525    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw4rn\" (UniqueName: \"kubernetes.io/projected/23988339-54db-4c05-b849-f669e303ab29-kube-api-access-bw4rn\") pod \"payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p\" (UID: \"23988339-54db-4c05-b849-f669e303ab29\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p"
Sep 10 17:27:50.806191 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.806173    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23988339-54db-4c05-b849-f669e303ab29-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p\" (UID: \"23988339-54db-4c05-b849-f669e303ab29\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p"
Sep 10 17:27:50.817145 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.817101    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw4rn\" (UniqueName: \"kubernetes.io/projected/23988339-54db-4c05-b849-f669e303ab29-kube-api-access-bw4rn\") pod \"payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p\" (UID: \"23988339-54db-4c05-b849-f669e303ab29\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p"
Sep 10 17:27:50.903991 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:50.903962    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p"
Sep 10 17:27:51.119768 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.119696    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"]
Sep 10 17:27:51.124445 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.124424    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.133261 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.133001    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-2da043-openshift-default-dockercfg-ssdsd\""
Sep 10 17:27:51.153863 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.153658    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"]
Sep 10 17:27:51.165397 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.165369    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:51.207360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.207335    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p"]
Sep 10 17:27:51.209259 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:27:51.209175    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23988339_54db_4c05_b849_f669e303ab29.slice/crio-b3c0c75ff73585fa1051500cda3b73cc4007c9cd45085e3901f31097de3136cd WatchSource:0}: Error finding container b3c0c75ff73585fa1051500cda3b73cc4007c9cd45085e3901f31097de3136cd: Status 404 returned error can't find the container with id b3c0c75ff73585fa1051500cda3b73cc4007c9cd45085e3901f31097de3136cd
Sep 10 17:27:51.209500 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.209468    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-credential-socket\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.209599 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.209556    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-istio-data\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.209716 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.209697    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e80813d-281d-4109-8e31-946428b49733-istiod-ca-cert\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.209792 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.209756    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-istio-envoy\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.209851 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.209799    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-workload-certs\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.209903 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.209849    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e80813d-281d-4109-8e31-946428b49733-istio-token\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.209903 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.209890    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e80813d-281d-4109-8e31-946428b49733-istio-podinfo\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.210006 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.209918    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-workload-socket\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.210006 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.209945    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24xf6\" (UniqueName: \"kubernetes.io/projected/0e80813d-281d-4109-8e31-946428b49733-kube-api-access-24xf6\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.310929 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.310901    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-istio-envoy\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.311116 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.310945    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-workload-certs\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.311116 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.310984    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e80813d-281d-4109-8e31-946428b49733-istio-token\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.311116 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.311029    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e80813d-281d-4109-8e31-946428b49733-istio-podinfo\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.311116 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.311055    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-workload-socket\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.311369 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.311308    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24xf6\" (UniqueName: \"kubernetes.io/projected/0e80813d-281d-4109-8e31-946428b49733-kube-api-access-24xf6\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.311369 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.311363    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-credential-socket\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.311476 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.311410    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-istio-data\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.311527 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.311510    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e80813d-281d-4109-8e31-946428b49733-istiod-ca-cert\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.311594 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.311561    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-workload-certs\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.311862 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.311835    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-credential-socket\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.312012 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.311866    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-istio-data\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.312012 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.311841    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-workload-socket\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.312397 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.312376    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e80813d-281d-4109-8e31-946428b49733-istiod-ca-cert\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.313510 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.313487    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-istio-envoy\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.314345 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.314325    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e80813d-281d-4109-8e31-946428b49733-istio-podinfo\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.320735 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.320711    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e80813d-281d-4109-8e31-946428b49733-istio-token\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.321299 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.321269    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24xf6\" (UniqueName: \"kubernetes.io/projected/0e80813d-281d-4109-8e31-946428b49733-kube-api-access-24xf6\") pod \"e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") " pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.429627 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.429598    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p" event={"ID":"23988339-54db-4c05-b849-f669e303ab29","Type":"ContainerStarted","Data":"b3c0c75ff73585fa1051500cda3b73cc4007c9cd45085e3901f31097de3136cd"}
Sep 10 17:27:51.429959 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.429666    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7926-7282da" podUID="c2c7c124-0f47-444b-a819-b26dd7064d61" containerName="test-curl-7926-7282da" containerID="cri-o://9e9c2959aca312bab446813a5919de3cdd6d6804fff8e79e3744aa006adb5026" gracePeriod=2
Sep 10 17:27:51.436449 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.436430    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:51.582258 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.582235    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7926-48bb84"
Sep 10 17:27:51.716338 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.716166    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kwgj\" (UniqueName: \"kubernetes.io/projected/94d6e2ee-3117-4e25-98bc-27dd22e1b19f-kube-api-access-7kwgj\") pod \"94d6e2ee-3117-4e25-98bc-27dd22e1b19f\" (UID: \"94d6e2ee-3117-4e25-98bc-27dd22e1b19f\") "
Sep 10 17:27:51.719293 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.719272    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94d6e2ee-3117-4e25-98bc-27dd22e1b19f-kube-api-access-7kwgj" (OuterVolumeSpecName: "kube-api-access-7kwgj") pod "94d6e2ee-3117-4e25-98bc-27dd22e1b19f" (UID: "94d6e2ee-3117-4e25-98bc-27dd22e1b19f"). InnerVolumeSpecName "kube-api-access-7kwgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:27:51.721563 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.721528    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"]
Sep 10 17:27:51.729163 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.729114    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:27:51.729258 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.729199    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:27:51.729258 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.729227    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:27:51.742722 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.742686    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:51.818041 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:51.818004    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kwgj\" (UniqueName: \"kubernetes.io/projected/94d6e2ee-3117-4e25-98bc-27dd22e1b19f-kube-api-access-7kwgj\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:27:52.439948 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:52.439910    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d" event={"ID":"0e80813d-281d-4109-8e31-946428b49733","Type":"ContainerStarted","Data":"235426d15ebeb9831e6240377c3a2b85df25517288f79c1be09de691125bb3d9"}
Sep 10 17:27:52.439948 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:52.439980    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d" event={"ID":"0e80813d-281d-4109-8e31-946428b49733","Type":"ContainerStarted","Data":"ffcd91b10796fb820d6d6c292dcac97a91185e67709aa25a34f815e032471a3b"}
Sep 10 17:27:52.441539 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:52.441509    2566 generic.go:356] "Generic (PLEG): container finished" podID="94d6e2ee-3117-4e25-98bc-27dd22e1b19f" containerID="9894cd8ed0af6810e3cfc58581036621bf115080c94504166ba12d90043a05fb" exitCode=137
Sep 10 17:27:52.441663 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:52.441567    2566 scope.go:117] "RemoveContainer" containerID="9894cd8ed0af6810e3cfc58581036621bf115080c94504166ba12d90043a05fb"
Sep 10 17:27:52.441663 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:52.441571    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7926-48bb84"
Sep 10 17:27:52.452591 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:52.452571    2566 scope.go:117] "RemoveContainer" containerID="9894cd8ed0af6810e3cfc58581036621bf115080c94504166ba12d90043a05fb"
Sep 10 17:27:52.452859 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:27:52.452838    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9894cd8ed0af6810e3cfc58581036621bf115080c94504166ba12d90043a05fb\": container with ID starting with 9894cd8ed0af6810e3cfc58581036621bf115080c94504166ba12d90043a05fb not found: ID does not exist" containerID="9894cd8ed0af6810e3cfc58581036621bf115080c94504166ba12d90043a05fb"
Sep 10 17:27:52.452946 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:52.452870    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9894cd8ed0af6810e3cfc58581036621bf115080c94504166ba12d90043a05fb"} err="failed to get container status \"9894cd8ed0af6810e3cfc58581036621bf115080c94504166ba12d90043a05fb\": rpc error: code = NotFound desc = could not find container \"9894cd8ed0af6810e3cfc58581036621bf115080c94504166ba12d90043a05fb\": container with ID starting with 9894cd8ed0af6810e3cfc58581036621bf115080c94504166ba12d90043a05fb not found: ID does not exist"
Sep 10 17:27:52.465018 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:52.464978    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d" podStartSLOduration=1.4649655959999999 podStartE2EDuration="1.464965596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:27:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:27:52.463007969 +0000 UTC m=+876.144180912" watchObservedRunningTime="2026-09-10 17:27:52.464965596 +0000 UTC m=+876.146138610"
Sep 10 17:27:52.465133 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:27:52.465096    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7926-48bb84\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" podUID="94d6e2ee-3117-4e25-98bc-27dd22e1b19f" pod="opendatahub/test-curl-7926-48bb84"
Sep 10 17:27:52.835906 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:52.835870    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94d6e2ee-3117-4e25-98bc-27dd22e1b19f" path="/var/lib/kubelet/pods/94d6e2ee-3117-4e25-98bc-27dd22e1b19f/volumes"
Sep 10 17:27:53.436880 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:53.436849    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:53.443556 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:53.443534    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:53.450597 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:53.450569    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p" event={"ID":"23988339-54db-4c05-b849-f669e303ab29","Type":"ContainerStarted","Data":"85dcd6cae7f9b930dafbc456a68788f875afeaf2af0e59f1d76a24aba526898e"}
Sep 10 17:27:53.450686 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:53.450622    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p"
Sep 10 17:27:53.451770 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:53.451754    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:53.452510 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:53.452493    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:27:53.497662 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:53.497615    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p" podStartSLOduration=1.710495213 podStartE2EDuration="3.497600285s" totalImagesPullingTime="1.787105072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:27:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:27:53.496378212 +0000 UTC m=+877.177551156" watchObservedRunningTime="2026-09-10 17:27:53.497600285 +0000 UTC m=+877.178773228"
Sep 10 17:27:53.604472 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:53.604447    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7926-7282da"
Sep 10 17:27:53.738403 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:53.738315    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9q59d\" (UniqueName: \"kubernetes.io/projected/c2c7c124-0f47-444b-a819-b26dd7064d61-kube-api-access-9q59d\") pod \"c2c7c124-0f47-444b-a819-b26dd7064d61\" (UID: \"c2c7c124-0f47-444b-a819-b26dd7064d61\") "
Sep 10 17:27:53.740351 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:53.740321    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2c7c124-0f47-444b-a819-b26dd7064d61-kube-api-access-9q59d" (OuterVolumeSpecName: "kube-api-access-9q59d") pod "c2c7c124-0f47-444b-a819-b26dd7064d61" (UID: "c2c7c124-0f47-444b-a819-b26dd7064d61"). InnerVolumeSpecName "kube-api-access-9q59d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:27:53.792191 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:53.792163    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:53.839549 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:53.839514    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9q59d\" (UniqueName: \"kubernetes.io/projected/c2c7c124-0f47-444b-a819-b26dd7064d61-kube-api-access-9q59d\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:27:53.891334 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:53.891304    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:54.002895 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:54.002856    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:54.107634 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:54.107604    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:54.411666 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:54.411572    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:54.460037 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:54.460002    2566 generic.go:356] "Generic (PLEG): container finished" podID="c2c7c124-0f47-444b-a819-b26dd7064d61" containerID="9e9c2959aca312bab446813a5919de3cdd6d6804fff8e79e3744aa006adb5026" exitCode=137
Sep 10 17:27:54.460037 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:54.460034    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7926-7282da"
Sep 10 17:27:54.460564 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:54.460095    2566 scope.go:117] "RemoveContainer" containerID="9e9c2959aca312bab446813a5919de3cdd6d6804fff8e79e3744aa006adb5026"
Sep 10 17:27:54.469716 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:54.469569    2566 scope.go:117] "RemoveContainer" containerID="9e9c2959aca312bab446813a5919de3cdd6d6804fff8e79e3744aa006adb5026"
Sep 10 17:27:54.469825 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:27:54.469808    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e9c2959aca312bab446813a5919de3cdd6d6804fff8e79e3744aa006adb5026\": container with ID starting with 9e9c2959aca312bab446813a5919de3cdd6d6804fff8e79e3744aa006adb5026 not found: ID does not exist" containerID="9e9c2959aca312bab446813a5919de3cdd6d6804fff8e79e3744aa006adb5026"
Sep 10 17:27:54.469865 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:54.469832    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e9c2959aca312bab446813a5919de3cdd6d6804fff8e79e3744aa006adb5026"} err="failed to get container status \"9e9c2959aca312bab446813a5919de3cdd6d6804fff8e79e3744aa006adb5026\": rpc error: code = NotFound desc = could not find container \"9e9c2959aca312bab446813a5919de3cdd6d6804fff8e79e3744aa006adb5026\": container with ID starting with 9e9c2959aca312bab446813a5919de3cdd6d6804fff8e79e3744aa006adb5026 not found: ID does not exist"
Sep 10 17:27:54.472699 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:27:54.472665    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7926-7282da\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" podUID="c2c7c124-0f47-444b-a819-b26dd7064d61" pod="opendatahub/test-curl-7926-7282da"
Sep 10 17:27:54.571783 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:54.571753    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:54.696500 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:54.696417    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:54.835602 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:54.835561    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2c7c124-0f47-444b-a819-b26dd7064d61" path="/var/lib/kubelet/pods/c2c7c124-0f47-444b-a819-b26dd7064d61/volumes"
Sep 10 17:27:55.306261 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:55.306226    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:56.600214 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.600180    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"]
Sep 10 17:27:56.605436 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.605411    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:27:56.612547 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.612521    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 17:27:56.612547 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.612537    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 17:27:56.612766 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.612521    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-2da043-serving-cert\""
Sep 10 17:27:56.613356 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.613338    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:27:56.622467 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.622448    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-2hfkp\""
Sep 10 17:27:56.668318 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.668291    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a70068e-8b2f-49f5-9893-80290b22b244-metrics-certs\") pod \"maas-api-e2e-autores-2da043-5858558c65-b6fvs\" (UID: \"2a70068e-8b2f-49f5-9893-80290b22b244\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:27:56.668466 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.668445    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a70068e-8b2f-49f5-9893-80290b22b244-openshift-service-ca\") pod \"maas-api-e2e-autores-2da043-5858558c65-b6fvs\" (UID: \"2a70068e-8b2f-49f5-9893-80290b22b244\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:27:56.668532 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.668516    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a70068e-8b2f-49f5-9893-80290b22b244-maas-api-tls\") pod \"maas-api-e2e-autores-2da043-5858558c65-b6fvs\" (UID: \"2a70068e-8b2f-49f5-9893-80290b22b244\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:27:56.668575 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.668547    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsqhr\" (UniqueName: \"kubernetes.io/projected/2a70068e-8b2f-49f5-9893-80290b22b244-kube-api-access-fsqhr\") pod \"maas-api-e2e-autores-2da043-5858558c65-b6fvs\" (UID: \"2a70068e-8b2f-49f5-9893-80290b22b244\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:27:56.672388 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.672363    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:56.703110 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.703079    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"]
Sep 10 17:27:56.769547 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.769510    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a70068e-8b2f-49f5-9893-80290b22b244-openshift-service-ca\") pod \"maas-api-e2e-autores-2da043-5858558c65-b6fvs\" (UID: \"2a70068e-8b2f-49f5-9893-80290b22b244\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:27:56.769732 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.769573    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a70068e-8b2f-49f5-9893-80290b22b244-maas-api-tls\") pod \"maas-api-e2e-autores-2da043-5858558c65-b6fvs\" (UID: \"2a70068e-8b2f-49f5-9893-80290b22b244\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:27:56.769732 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.769596    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsqhr\" (UniqueName: \"kubernetes.io/projected/2a70068e-8b2f-49f5-9893-80290b22b244-kube-api-access-fsqhr\") pod \"maas-api-e2e-autores-2da043-5858558c65-b6fvs\" (UID: \"2a70068e-8b2f-49f5-9893-80290b22b244\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:27:56.769732 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.769624    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a70068e-8b2f-49f5-9893-80290b22b244-metrics-certs\") pod \"maas-api-e2e-autores-2da043-5858558c65-b6fvs\" (UID: \"2a70068e-8b2f-49f5-9893-80290b22b244\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:27:56.769899 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:27:56.769746    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-2da043-serving-cert: secret "maas-api-e2e-autores-2da043-serving-cert" not found
Sep 10 17:27:56.769899 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:27:56.769825    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a70068e-8b2f-49f5-9893-80290b22b244-maas-api-tls podName:2a70068e-8b2f-49f5-9893-80290b22b244 nodeName:}" failed. No retries permitted until 2026-09-10 17:27:57.269806042 +0000 UTC m=+880.950978978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2a70068e-8b2f-49f5-9893-80290b22b244-maas-api-tls") pod "maas-api-e2e-autores-2da043-5858558c65-b6fvs" (UID: "2a70068e-8b2f-49f5-9893-80290b22b244") : secret "maas-api-e2e-autores-2da043-serving-cert" not found
Sep 10 17:27:56.770173 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.770149    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a70068e-8b2f-49f5-9893-80290b22b244-openshift-service-ca\") pod \"maas-api-e2e-autores-2da043-5858558c65-b6fvs\" (UID: \"2a70068e-8b2f-49f5-9893-80290b22b244\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:27:56.772502 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.772479    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a70068e-8b2f-49f5-9893-80290b22b244-metrics-certs\") pod \"maas-api-e2e-autores-2da043-5858558c65-b6fvs\" (UID: \"2a70068e-8b2f-49f5-9893-80290b22b244\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:27:56.775203 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.775172    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:56.780600 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.780571    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsqhr\" (UniqueName: \"kubernetes.io/projected/2a70068e-8b2f-49f5-9893-80290b22b244-kube-api-access-fsqhr\") pod \"maas-api-e2e-autores-2da043-5858558c65-b6fvs\" (UID: \"2a70068e-8b2f-49f5-9893-80290b22b244\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:27:56.948762 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:56.948655    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:57.080320 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:57.080289    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:57.276033 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:57.275963    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a70068e-8b2f-49f5-9893-80290b22b244-maas-api-tls\") pod \"maas-api-e2e-autores-2da043-5858558c65-b6fvs\" (UID: \"2a70068e-8b2f-49f5-9893-80290b22b244\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:27:57.278527 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:57.278497    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a70068e-8b2f-49f5-9893-80290b22b244-maas-api-tls\") pod \"maas-api-e2e-autores-2da043-5858558c65-b6fvs\" (UID: \"2a70068e-8b2f-49f5-9893-80290b22b244\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:27:57.517594 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:57.517558    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:27:57.835287 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:57.835212    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"]
Sep 10 17:27:57.838421 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:27:57.838390    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a70068e_8b2f_49f5_9893_80290b22b244.slice/crio-1006aba4a07cc66201b8153639ef1f48cf64b086a1803639bfdf6455a52809bc WatchSource:0}: Error finding container 1006aba4a07cc66201b8153639ef1f48cf64b086a1803639bfdf6455a52809bc: Status 404 returned error can't find the container with id 1006aba4a07cc66201b8153639ef1f48cf64b086a1803639bfdf6455a52809bc
Sep 10 17:27:57.869894 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:57.869860    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:58.015014 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.014979    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:58.105053 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.104976    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf"]
Sep 10 17:27:58.110337 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.110310    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf"
Sep 10 17:27:58.112934 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.112909    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-2da043-dockercfg-7l7fw\""
Sep 10 17:27:58.113227 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.113210    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-2da043\""
Sep 10 17:27:58.124225 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.124199    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf"]
Sep 10 17:27:58.186472 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.186436    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97ba04cb-05e1-418a-8dac-e5b951f997ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-2da043-756b789474-npbjf\" (UID: \"97ba04cb-05e1-418a-8dac-e5b951f997ae\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf"
Sep 10 17:27:58.186638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.186559    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvcvc\" (UniqueName: \"kubernetes.io/projected/97ba04cb-05e1-418a-8dac-e5b951f997ae-kube-api-access-mvcvc\") pod \"payload-pre-processing-e2e-autores-2da043-756b789474-npbjf\" (UID: \"97ba04cb-05e1-418a-8dac-e5b951f997ae\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf"
Sep 10 17:27:58.287522 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.287484    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97ba04cb-05e1-418a-8dac-e5b951f997ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-2da043-756b789474-npbjf\" (UID: \"97ba04cb-05e1-418a-8dac-e5b951f997ae\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf"
Sep 10 17:27:58.287787 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.287585    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvcvc\" (UniqueName: \"kubernetes.io/projected/97ba04cb-05e1-418a-8dac-e5b951f997ae-kube-api-access-mvcvc\") pod \"payload-pre-processing-e2e-autores-2da043-756b789474-npbjf\" (UID: \"97ba04cb-05e1-418a-8dac-e5b951f997ae\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf"
Sep 10 17:27:58.288387 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.288362    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97ba04cb-05e1-418a-8dac-e5b951f997ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-2da043-756b789474-npbjf\" (UID: \"97ba04cb-05e1-418a-8dac-e5b951f997ae\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf"
Sep 10 17:27:58.302332 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.301830    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvcvc\" (UniqueName: \"kubernetes.io/projected/97ba04cb-05e1-418a-8dac-e5b951f997ae-kube-api-access-mvcvc\") pod \"payload-pre-processing-e2e-autores-2da043-756b789474-npbjf\" (UID: \"97ba04cb-05e1-418a-8dac-e5b951f997ae\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf"
Sep 10 17:27:58.423990 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.423908    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf"
Sep 10 17:27:58.490730 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.490454    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs" event={"ID":"2a70068e-8b2f-49f5-9893-80290b22b244","Type":"ContainerStarted","Data":"1006aba4a07cc66201b8153639ef1f48cf64b086a1803639bfdf6455a52809bc"}
Sep 10 17:27:58.660254 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.660220    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"]
Sep 10 17:27:58.665391 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.665370    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"
Sep 10 17:27:58.670946 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.670924    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e52f3775-serving-cert\""
Sep 10 17:27:58.681349 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.681323    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"]
Sep 10 17:27:58.722457 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.722424    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:58.740758 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.740732    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf"]
Sep 10 17:27:58.742569 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:27:58.742542    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97ba04cb_05e1_418a_8dac_e5b951f997ae.slice/crio-c2449820f23ea0ad8b68e7337d15ff787c6a61ed2df2616bb3c88ee46d4fbc81 WatchSource:0}: Error finding container c2449820f23ea0ad8b68e7337d15ff787c6a61ed2df2616bb3c88ee46d4fbc81: Status 404 returned error can't find the container with id c2449820f23ea0ad8b68e7337d15ff787c6a61ed2df2616bb3c88ee46d4fbc81
Sep 10 17:27:58.793964 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.793681    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjd9m\" (UniqueName: \"kubernetes.io/projected/66e7ae03-4985-4800-a11c-fc7676d31c13-kube-api-access-kjd9m\") pod \"maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh\" (UID: \"66e7ae03-4985-4800-a11c-fc7676d31c13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"
Sep 10 17:27:58.793964 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.793731    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66e7ae03-4985-4800-a11c-fc7676d31c13-metrics-certs\") pod \"maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh\" (UID: \"66e7ae03-4985-4800-a11c-fc7676d31c13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"
Sep 10 17:27:58.793964 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.793772    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66e7ae03-4985-4800-a11c-fc7676d31c13-openshift-service-ca\") pod \"maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh\" (UID: \"66e7ae03-4985-4800-a11c-fc7676d31c13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"
Sep 10 17:27:58.793964 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.793891    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66e7ae03-4985-4800-a11c-fc7676d31c13-maas-api-tls\") pod \"maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh\" (UID: \"66e7ae03-4985-4800-a11c-fc7676d31c13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"
Sep 10 17:27:58.829341 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.829299    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:58.895065 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.895031    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66e7ae03-4985-4800-a11c-fc7676d31c13-maas-api-tls\") pod \"maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh\" (UID: \"66e7ae03-4985-4800-a11c-fc7676d31c13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"
Sep 10 17:27:58.895533 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.895160    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjd9m\" (UniqueName: \"kubernetes.io/projected/66e7ae03-4985-4800-a11c-fc7676d31c13-kube-api-access-kjd9m\") pod \"maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh\" (UID: \"66e7ae03-4985-4800-a11c-fc7676d31c13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"
Sep 10 17:27:58.895533 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.895195    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66e7ae03-4985-4800-a11c-fc7676d31c13-metrics-certs\") pod \"maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh\" (UID: \"66e7ae03-4985-4800-a11c-fc7676d31c13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"
Sep 10 17:27:58.895533 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.895228    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66e7ae03-4985-4800-a11c-fc7676d31c13-openshift-service-ca\") pod \"maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh\" (UID: \"66e7ae03-4985-4800-a11c-fc7676d31c13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"
Sep 10 17:27:58.895939 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.895913    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66e7ae03-4985-4800-a11c-fc7676d31c13-openshift-service-ca\") pod \"maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh\" (UID: \"66e7ae03-4985-4800-a11c-fc7676d31c13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"
Sep 10 17:27:58.898197 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.898171    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66e7ae03-4985-4800-a11c-fc7676d31c13-metrics-certs\") pod \"maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh\" (UID: \"66e7ae03-4985-4800-a11c-fc7676d31c13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"
Sep 10 17:27:58.898835 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.898813    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66e7ae03-4985-4800-a11c-fc7676d31c13-maas-api-tls\") pod \"maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh\" (UID: \"66e7ae03-4985-4800-a11c-fc7676d31c13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"
Sep 10 17:27:58.908770 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.908722    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjd9m\" (UniqueName: \"kubernetes.io/projected/66e7ae03-4985-4800-a11c-fc7676d31c13-kube-api-access-kjd9m\") pod \"maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh\" (UID: \"66e7ae03-4985-4800-a11c-fc7676d31c13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"
Sep 10 17:27:58.926441 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.926330    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:58.981825 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:58.981790    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"
Sep 10 17:27:59.270275 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:59.270251    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"]
Sep 10 17:27:59.342142 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:59.342086    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:59.444813 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:59.444776    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:27:59.498851 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:59.498782    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf" event={"ID":"97ba04cb-05e1-418a-8dac-e5b951f997ae","Type":"ContainerStarted","Data":"4d664187abcf3d0fea1bb7240ca1f37c3dd8f352fd65357170a92e24383267bf"}
Sep 10 17:27:59.498851 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:59.498837    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf" event={"ID":"97ba04cb-05e1-418a-8dac-e5b951f997ae","Type":"ContainerStarted","Data":"c2449820f23ea0ad8b68e7337d15ff787c6a61ed2df2616bb3c88ee46d4fbc81"}
Sep 10 17:27:59.499076 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:59.498967    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf"
Sep 10 17:27:59.537212 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:59.537052    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf" podStartSLOduration=1.537032854 podStartE2EDuration="1.537032854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:27:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:27:59.535185853 +0000 UTC m=+883.216358824" watchObservedRunningTime="2026-09-10 17:27:59.537032854 +0000 UTC m=+883.218205798"
Sep 10 17:27:59.983205 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:27:59.983097    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:00.252627 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:28:00.252580    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66e7ae03_4985_4800_a11c_fc7676d31c13.slice/crio-82fe8e72c7bbbb42402dbe2d8f0955c467481cbf675d7f8b3b27e5f152e953d1 WatchSource:0}: Error finding container 82fe8e72c7bbbb42402dbe2d8f0955c467481cbf675d7f8b3b27e5f152e953d1: Status 404 returned error can't find the container with id 82fe8e72c7bbbb42402dbe2d8f0955c467481cbf675d7f8b3b27e5f152e953d1
Sep 10 17:28:00.506030 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:00.505947    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh" event={"ID":"66e7ae03-4985-4800-a11c-fc7676d31c13","Type":"ContainerStarted","Data":"82fe8e72c7bbbb42402dbe2d8f0955c467481cbf675d7f8b3b27e5f152e953d1"}
Sep 10 17:28:00.507564 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:00.507527    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs" event={"ID":"2a70068e-8b2f-49f5-9893-80290b22b244","Type":"ContainerStarted","Data":"0ce7b1c1e014c89fe4ff18890bcc5e7b717a5576c2b4bdec998b9dbdf05d5dd4"}
Sep 10 17:28:00.507785 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:00.507761    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:28:00.533654 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:00.533607    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs" podStartSLOduration=2.068543967 podStartE2EDuration="4.533593366s" totalImagesPullingTime="2.465049399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:27:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:28:00.532393798 +0000 UTC m=+884.213566739" watchObservedRunningTime="2026-09-10 17:28:00.533593366 +0000 UTC m=+884.214766309"
Sep 10 17:28:00.879758 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:00.879670    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:01.252557 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:01.252499    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:01.516560 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:01.516471    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh" event={"ID":"66e7ae03-4985-4800-a11c-fc7676d31c13","Type":"ContainerStarted","Data":"4b457b4edffa7f724079d0c9b3f227ce706934f09063905cbee2e5881912e2ca"}
Sep 10 17:28:01.516560 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:01.516532    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"
Sep 10 17:28:01.538272 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:01.538219    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh" podStartSLOduration=3.238724708 podStartE2EDuration="3.53820325s" totalImagesPullingTime="299.478542ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:27:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:28:01.536912157 +0000 UTC m=+885.218085101" watchObservedRunningTime="2026-09-10 17:28:01.53820325 +0000 UTC m=+885.219376217"
Sep 10 17:28:02.145328 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:02.145297    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:02.786323 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:02.786284    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:02.937770 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:02.937735    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:03.118926 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.118832    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:03.242905 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.242868    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:03.339852 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.339815    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:03.504549 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.504526    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"]
Sep 10 17:28:03.507094 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.507053    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:03.508220 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.508201    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.511860 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.511840    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-15ece2-openshift-default-dockercfg-t2f8d\""
Sep 10 17:28:03.530814 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.530785    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"]
Sep 10 17:28:03.646992 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.646962    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-credential-socket\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.646992 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.646995    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-workload-certs\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.647220 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.647018    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfm9x\" (UniqueName: \"kubernetes.io/projected/e0bd13d0-885d-4044-8425-6fe48e338199-kube-api-access-kfm9x\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.647220 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.647090    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0bd13d0-885d-4044-8425-6fe48e338199-istio-token\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.647220 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.647156    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-istio-data\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.647220 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.647212    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0bd13d0-885d-4044-8425-6fe48e338199-istio-podinfo\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.647349 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.647242    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0bd13d0-885d-4044-8425-6fe48e338199-istiod-ca-cert\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.647349 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.647275    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-istio-envoy\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.647349 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.647333    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-workload-socket\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.687620 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.687589    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:03.748874 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.748833    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0bd13d0-885d-4044-8425-6fe48e338199-istio-podinfo\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.749052 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.748877    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0bd13d0-885d-4044-8425-6fe48e338199-istiod-ca-cert\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.749052 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.749024    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-istio-envoy\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.749216 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.749165    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-workload-socket\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.749272 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.749222    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-credential-socket\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.749272 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.749251    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-workload-certs\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.749474 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.749453    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfm9x\" (UniqueName: \"kubernetes.io/projected/e0bd13d0-885d-4044-8425-6fe48e338199-kube-api-access-kfm9x\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.749581 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.749555    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-credential-socket\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.749645 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.749610    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-workload-certs\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.749702 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.749669    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0bd13d0-885d-4044-8425-6fe48e338199-istiod-ca-cert\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.749767 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.749722    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0bd13d0-885d-4044-8425-6fe48e338199-istio-token\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.749821 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.749790    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-istio-data\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.749821 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.749804    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-workload-socket\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.750158 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.750114    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-istio-data\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.751372 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.751355    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-istio-envoy\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.751843 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.751818    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0bd13d0-885d-4044-8425-6fe48e338199-istio-podinfo\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.773361 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.773309    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfm9x\" (UniqueName: \"kubernetes.io/projected/e0bd13d0-885d-4044-8425-6fe48e338199-kube-api-access-kfm9x\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.776556 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.776533    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0bd13d0-885d-4044-8425-6fe48e338199-istio-token\") pod \"e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") " pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.827502 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.827471    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:03.949257 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:03.949220    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:04.113022 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:04.112976    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"]
Sep 10 17:28:04.115231 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:28:04.115205    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0bd13d0_885d_4044_8425_6fe48e338199.slice/crio-6170a7c61d1c05b52fd5beb458daebd88ceeede190d93633cb6aa114aebdd73c WatchSource:0}: Error finding container 6170a7c61d1c05b52fd5beb458daebd88ceeede190d93633cb6aa114aebdd73c: Status 404 returned error can't find the container with id 6170a7c61d1c05b52fd5beb458daebd88ceeede190d93633cb6aa114aebdd73c
Sep 10 17:28:04.117658 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:04.117627    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:28:04.117750 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:04.117692    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:28:04.117750 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:04.117721    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:28:04.352995 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:04.352950    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:04.462818 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:04.462733    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p"
Sep 10 17:28:04.541684 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:04.541648    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw" event={"ID":"e0bd13d0-885d-4044-8425-6fe48e338199","Type":"ContainerStarted","Data":"167a8b2e4b78ef234b7c1c477ca74bcc0f32f78450640e85c044dfc9f281fc72"}
Sep 10 17:28:04.541885 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:04.541757    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw" event={"ID":"e0bd13d0-885d-4044-8425-6fe48e338199","Type":"ContainerStarted","Data":"6170a7c61d1c05b52fd5beb458daebd88ceeede190d93633cb6aa114aebdd73c"}
Sep 10 17:28:04.571757 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:04.571711    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw" podStartSLOduration=1.571696174 podStartE2EDuration="1.571696174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:28:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:28:04.569356266 +0000 UTC m=+888.250529219" watchObservedRunningTime="2026-09-10 17:28:04.571696174 +0000 UTC m=+888.252869194"
Sep 10 17:28:04.836633 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:04.836595    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:05.336908 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:05.336873    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:05.833836 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:05.833805    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:05.977797 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:05.977757    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:06.377362 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:06.377332    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:06.521674 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:06.521648    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:28:06.554738 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:06.554701    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:06.555733 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:06.555712    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:06.624149 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:06.624087    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:06.791982 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:06.791947    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:07.529104 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:07.529073    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"
Sep 10 17:28:07.806467 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:07.806370    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:08.291904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:08.291866    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:09.107528 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:09.107445    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:09.267610 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:09.267573    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:10.404439 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:10.404399    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:10.509709 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:10.509676    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf"
Sep 10 17:28:11.474933 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:11.474899    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:12.800097 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:12.800061    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:12.884790 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:12.884757    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:13.049647 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:13.049608    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:13.403234 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:13.403183    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:13.498491 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:13.498456    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:14.195253 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:14.195218    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:15.160396 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:15.160350    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:16.169801 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:16.169767    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:17.161928 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:17.161898    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:17.630781 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:17.630750    2566 generic.go:356] "Generic (PLEG): container finished" podID="0e80813d-281d-4109-8e31-946428b49733" containerID="235426d15ebeb9831e6240377c3a2b85df25517288f79c1be09de691125bb3d9" exitCode=0
Sep 10 17:28:17.630781 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:17.630786    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d" event={"ID":"0e80813d-281d-4109-8e31-946428b49733","Type":"ContainerDied","Data":"235426d15ebeb9831e6240377c3a2b85df25517288f79c1be09de691125bb3d9"}
Sep 10 17:28:17.631254 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:17.631235    2566 scope.go:117] "RemoveContainer" containerID="235426d15ebeb9831e6240377c3a2b85df25517288f79c1be09de691125bb3d9"
Sep 10 17:28:17.632275 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:17.632245    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:28:17.632398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:17.632320    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:28:17.632398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:17.632363    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:28:17.991992 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:17.991960    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:18.642862 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:18.642822    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d" event={"ID":"0e80813d-281d-4109-8e31-946428b49733","Type":"ContainerStarted","Data":"7bc55f1fc6ec0caa8b47350ee809d3c8db24b324be787423ba6d56716de267df"}
Sep 10 17:28:19.189814 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:19.189777    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:19.437933 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:19.437890    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:28:19.438911 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:19.438874    2566 patch_prober.go:28] interesting pod/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 10 17:28:19.439174 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:19.439114    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d" podUID="0e80813d-281d-4109-8e31-946428b49733" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 10 17:28:20.175694 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:20.175658    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:20.436944 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:20.436846    2566 patch_prober.go:28] interesting pod/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 10 17:28:20.437895 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:20.437860    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d" podUID="0e80813d-281d-4109-8e31-946428b49733" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 10 17:28:21.436832 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:21.436783    2566 patch_prober.go:28] interesting pod/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 10 17:28:21.437364 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:21.436854    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d" podUID="0e80813d-281d-4109-8e31-946428b49733" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 10 17:28:22.437091 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:22.437053    2566 patch_prober.go:28] interesting pod/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 10 17:28:22.437673 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:22.437642    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d" podUID="0e80813d-281d-4109-8e31-946428b49733" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 10 17:28:22.755388 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:22.755352    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"]
Sep 10 17:28:22.755876 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:22.755822    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh" podUID="66e7ae03-4985-4800-a11c-fc7676d31c13" containerName="maas-api" containerID="cri-o://4b457b4edffa7f724079d0c9b3f227ce706934f09063905cbee2e5881912e2ca" gracePeriod=30
Sep 10 17:28:22.787489 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:22.784673    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:22.888369 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:22.888258    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:22.980081 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:22.980008    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:23.072046 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.072016    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:23.157287 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.157208    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:23.194149 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.193511    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"
Sep 10 17:28:23.278201 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.275580    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:23.285477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.285442    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjd9m\" (UniqueName: \"kubernetes.io/projected/66e7ae03-4985-4800-a11c-fc7676d31c13-kube-api-access-kjd9m\") pod \"66e7ae03-4985-4800-a11c-fc7676d31c13\" (UID: \"66e7ae03-4985-4800-a11c-fc7676d31c13\") "
Sep 10 17:28:23.285644 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.285512    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66e7ae03-4985-4800-a11c-fc7676d31c13-maas-api-tls\") pod \"66e7ae03-4985-4800-a11c-fc7676d31c13\" (UID: \"66e7ae03-4985-4800-a11c-fc7676d31c13\") "
Sep 10 17:28:23.285644 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.285532    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66e7ae03-4985-4800-a11c-fc7676d31c13-metrics-certs\") pod \"66e7ae03-4985-4800-a11c-fc7676d31c13\" (UID: \"66e7ae03-4985-4800-a11c-fc7676d31c13\") "
Sep 10 17:28:23.285644 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.285627    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66e7ae03-4985-4800-a11c-fc7676d31c13-openshift-service-ca\") pod \"66e7ae03-4985-4800-a11c-fc7676d31c13\" (UID: \"66e7ae03-4985-4800-a11c-fc7676d31c13\") "
Sep 10 17:28:23.286300 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.286271    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66e7ae03-4985-4800-a11c-fc7676d31c13-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "66e7ae03-4985-4800-a11c-fc7676d31c13" (UID: "66e7ae03-4985-4800-a11c-fc7676d31c13"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:28:23.288004 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.287973    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66e7ae03-4985-4800-a11c-fc7676d31c13-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "66e7ae03-4985-4800-a11c-fc7676d31c13" (UID: "66e7ae03-4985-4800-a11c-fc7676d31c13"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:28:23.288100 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.288005    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66e7ae03-4985-4800-a11c-fc7676d31c13-kube-api-access-kjd9m" (OuterVolumeSpecName: "kube-api-access-kjd9m") pod "66e7ae03-4985-4800-a11c-fc7676d31c13" (UID: "66e7ae03-4985-4800-a11c-fc7676d31c13"). InnerVolumeSpecName "kube-api-access-kjd9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:28:23.288216 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.288195    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66e7ae03-4985-4800-a11c-fc7676d31c13-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "66e7ae03-4985-4800-a11c-fc7676d31c13" (UID: "66e7ae03-4985-4800-a11c-fc7676d31c13"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:28:23.386513 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.386476    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66e7ae03-4985-4800-a11c-fc7676d31c13-openshift-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:28:23.386513 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.386506    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjd9m\" (UniqueName: \"kubernetes.io/projected/66e7ae03-4985-4800-a11c-fc7676d31c13-kube-api-access-kjd9m\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:28:23.386722 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.386528    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66e7ae03-4985-4800-a11c-fc7676d31c13-maas-api-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:28:23.386722 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.386536    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66e7ae03-4985-4800-a11c-fc7676d31c13-metrics-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:28:23.437439 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.437403    2566 patch_prober.go:28] interesting pod/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 10 17:28:23.437980 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.437948    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d" podUID="0e80813d-281d-4109-8e31-946428b49733" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 10 17:28:23.452161 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.452114    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:28:23.681282 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.681164    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:23.683795 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.683762    2566 generic.go:356] "Generic (PLEG): container finished" podID="66e7ae03-4985-4800-a11c-fc7676d31c13" containerID="4b457b4edffa7f724079d0c9b3f227ce706934f09063905cbee2e5881912e2ca" exitCode=0
Sep 10 17:28:23.683941 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.683803    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh" event={"ID":"66e7ae03-4985-4800-a11c-fc7676d31c13","Type":"ContainerDied","Data":"4b457b4edffa7f724079d0c9b3f227ce706934f09063905cbee2e5881912e2ca"}
Sep 10 17:28:23.683941 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.683819    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"
Sep 10 17:28:23.683941 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.683839    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh" event={"ID":"66e7ae03-4985-4800-a11c-fc7676d31c13","Type":"ContainerDied","Data":"82fe8e72c7bbbb42402dbe2d8f0955c467481cbf675d7f8b3b27e5f152e953d1"}
Sep 10 17:28:23.683941 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.683860    2566 scope.go:117] "RemoveContainer" containerID="4b457b4edffa7f724079d0c9b3f227ce706934f09063905cbee2e5881912e2ca"
Sep 10 17:28:23.697983 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.697967    2566 scope.go:117] "RemoveContainer" containerID="4b457b4edffa7f724079d0c9b3f227ce706934f09063905cbee2e5881912e2ca"
Sep 10 17:28:23.698299 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:28:23.698265    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b457b4edffa7f724079d0c9b3f227ce706934f09063905cbee2e5881912e2ca\": container with ID starting with 4b457b4edffa7f724079d0c9b3f227ce706934f09063905cbee2e5881912e2ca not found: ID does not exist" containerID="4b457b4edffa7f724079d0c9b3f227ce706934f09063905cbee2e5881912e2ca"
Sep 10 17:28:23.698389 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.698302    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b457b4edffa7f724079d0c9b3f227ce706934f09063905cbee2e5881912e2ca"} err="failed to get container status \"4b457b4edffa7f724079d0c9b3f227ce706934f09063905cbee2e5881912e2ca\": rpc error: code = NotFound desc = could not find container \"4b457b4edffa7f724079d0c9b3f227ce706934f09063905cbee2e5881912e2ca\": container with ID starting with 4b457b4edffa7f724079d0c9b3f227ce706934f09063905cbee2e5881912e2ca not found: ID does not exist"
Sep 10 17:28:23.768149 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.768079    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"]
Sep 10 17:28:23.774672 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.774642    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e52f3775-79d5897df4-vb6wh"]
Sep 10 17:28:23.825784 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:23.825742    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:24.093773 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.093736    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:24.318941 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.318894    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"]
Sep 10 17:28:24.320600 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.320578    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66e7ae03-4985-4800-a11c-fc7676d31c13" containerName="maas-api"
Sep 10 17:28:24.320778 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.320764    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="66e7ae03-4985-4800-a11c-fc7676d31c13" containerName="maas-api"
Sep 10 17:28:24.321044 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.321029    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="66e7ae03-4985-4800-a11c-fc7676d31c13" containerName="maas-api"
Sep 10 17:28:24.328218 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.324250    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:24.337379 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.337355    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.338484 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.338461    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"]
Sep 10 17:28:24.340412 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.340393    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-2d96ccec-gw-openshift-default-dockercfg-hznd9\""
Sep 10 17:28:24.396493 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.396456    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-token\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.396678 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.396562    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-workload-certs\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.396678 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.396587    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-credential-socket\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.396678 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.396612    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-workload-socket\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.396678 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.396638    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istiod-ca-cert\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.396678 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.396665    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-data\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.396966 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.396755    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-podinfo\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.396966 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.396787    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-envoy\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.396966 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.396803    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dvbb\" (UniqueName: \"kubernetes.io/projected/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-kube-api-access-7dvbb\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.443105 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.443068    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:28:24.444347 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.444327    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:28:24.497730 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.497698    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-podinfo\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.497951 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.497913    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-envoy\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.498058 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.497958    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dvbb\" (UniqueName: \"kubernetes.io/projected/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-kube-api-access-7dvbb\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.498058 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.498008    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-token\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.498201 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.498102    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-workload-certs\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.498269 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.498194    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-credential-socket\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.498269 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.498239    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-workload-socket\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.498414 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.498277    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istiod-ca-cert\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.498414 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.498308    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-data\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.498574 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.498548    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-workload-certs\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.498675 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.498598    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-credential-socket\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.499313 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.499287    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-workload-socket\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.499403 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.499344    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istiod-ca-cert\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.499465 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.499304    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-data\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.500664 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.500639    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-podinfo\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.500664 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.500659    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-envoy\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.518527 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.518492    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-token\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.520791 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.520761    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dvbb\" (UniqueName: \"kubernetes.io/projected/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-kube-api-access-7dvbb\") pod \"e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") " pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.655396 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.655309    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:24.792144 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.792079    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:24.839655 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.839576    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66e7ae03-4985-4800-a11c-fc7676d31c13" path="/var/lib/kubelet/pods/66e7ae03-4985-4800-a11c-fc7676d31c13/volumes"
Sep 10 17:28:24.951209 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.951180    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"]
Sep 10 17:28:24.952002 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:28:24.951972    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52eaae0f_c7d1_4458_9a80_2e4e5ef8c07d.slice/crio-bcb53144a938d930e551a88d9bde754e595f4ab7bd0def2cc0f6ff1db7cd9eec WatchSource:0}: Error finding container bcb53144a938d930e551a88d9bde754e595f4ab7bd0def2cc0f6ff1db7cd9eec: Status 404 returned error can't find the container with id bcb53144a938d930e551a88d9bde754e595f4ab7bd0def2cc0f6ff1db7cd9eec
Sep 10 17:28:24.954784 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.954745    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:28:24.954964 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.954939    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:28:24.955426 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:24.955277    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:28:25.173096 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:25.173061    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:25.700061 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:25.700021    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh" event={"ID":"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d","Type":"ContainerStarted","Data":"71fa5bfcdb01f2ff68b94daddfd5aa4b7a437c86e4f8ba1e212c6205f3e9da61"}
Sep 10 17:28:25.700520 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:25.700076    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh" event={"ID":"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d","Type":"ContainerStarted","Data":"bcb53144a938d930e551a88d9bde754e595f4ab7bd0def2cc0f6ff1db7cd9eec"}
Sep 10 17:28:25.733728 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:25.733676    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh" podStartSLOduration=1.733661221 podStartE2EDuration="1.733661221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:28:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:28:25.731074028 +0000 UTC m=+909.412246971" watchObservedRunningTime="2026-09-10 17:28:25.733661221 +0000 UTC m=+909.414834163"
Sep 10 17:28:26.095308 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:26.095269    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:26.303830 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:26.303785    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:26.655755 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:26.655723    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:26.661495 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:26.661469    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:26.706475 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:26.706442    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:26.707793 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:26.707777    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:27.041102 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:27.041062    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:27.326772 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:27.326691    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:27.419611 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:27.419573    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:28.116656 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:28.116625    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:28.720188 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:28.720152    2566 generic.go:356] "Generic (PLEG): container finished" podID="e0bd13d0-885d-4044-8425-6fe48e338199" containerID="167a8b2e4b78ef234b7c1c477ca74bcc0f32f78450640e85c044dfc9f281fc72" exitCode=0
Sep 10 17:28:28.720355 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:28.720217    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw" event={"ID":"e0bd13d0-885d-4044-8425-6fe48e338199","Type":"ContainerDied","Data":"167a8b2e4b78ef234b7c1c477ca74bcc0f32f78450640e85c044dfc9f281fc72"}
Sep 10 17:28:28.720833 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:28.720817    2566 scope.go:117] "RemoveContainer" containerID="167a8b2e4b78ef234b7c1c477ca74bcc0f32f78450640e85c044dfc9f281fc72"
Sep 10 17:28:28.721991 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:28.721925    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:28:28.722065 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:28.722048    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:28:28.722112 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:28.722097    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:28:29.156581 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:29.156535    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:29.746506 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:29.746473    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw" event={"ID":"e0bd13d0-885d-4044-8425-6fe48e338199","Type":"ContainerStarted","Data":"9b4ca983e632cc4b7d49225787e09c335097137232ebae6e328f902265241f5f"}
Sep 10 17:28:29.829368 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:29.829328    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:29.829602 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:29.829572    2566 patch_prober.go:28] interesting pod/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 10 17:28:29.829684 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:29.829650    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw" podUID="e0bd13d0-885d-4044-8425-6fe48e338199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 10 17:28:30.155771 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.155692    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"]
Sep 10 17:28:30.159574 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.159553    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:28:30.162334 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.162313    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-2d96ccec-serving-cert\""
Sep 10 17:28:30.179448 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.179094    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"]
Sep 10 17:28:30.247016 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.246987    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:30.259430 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.259033    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpdqh\" (UniqueName: \"kubernetes.io/projected/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-kube-api-access-jpdqh\") pod \"maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n\" (UID: \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:28:30.259430 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.259120    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-metrics-certs\") pod \"maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n\" (UID: \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:28:30.259430 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.259275    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-maas-api-tls\") pod \"maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n\" (UID: \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:28:30.259430 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.259322    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-openshift-service-ca\") pod \"maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n\" (UID: \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:28:30.337149 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.335910    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:30.360852 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.360804    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-metrics-certs\") pod \"maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n\" (UID: \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:28:30.361041 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.360893    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-maas-api-tls\") pod \"maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n\" (UID: \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:28:30.361041 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.360927    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-openshift-service-ca\") pod \"maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n\" (UID: \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:28:30.361041 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.360979    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpdqh\" (UniqueName: \"kubernetes.io/projected/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-kube-api-access-jpdqh\") pod \"maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n\" (UID: \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:28:30.361041 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:28:30.361025    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-serving-cert: secret "maas-api-e2e-migrate-2d96ccec-serving-cert" not found
Sep 10 17:28:30.361298 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:28:30.361111    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-maas-api-tls podName:6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0 nodeName:}" failed. No retries permitted until 2026-09-10 17:28:30.861088163 +0000 UTC m=+914.542261089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-maas-api-tls") pod "maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n" (UID: "6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0") : secret "maas-api-e2e-migrate-2d96ccec-serving-cert" not found
Sep 10 17:28:30.361901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.361879    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-openshift-service-ca\") pod \"maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n\" (UID: \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:28:30.363318 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.363297    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-metrics-certs\") pod \"maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n\" (UID: \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:28:30.379103 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.379084    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpdqh\" (UniqueName: \"kubernetes.io/projected/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-kube-api-access-jpdqh\") pod \"maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n\" (UID: \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:28:30.566890 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.566857    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:30.831206 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.829489    2566 patch_prober.go:28] interesting pod/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 10 17:28:30.831206 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.829554    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw" podUID="e0bd13d0-885d-4044-8425-6fe48e338199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 10 17:28:30.841265 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.841225    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:30.865330 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.865301    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-maas-api-tls\") pod \"maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n\" (UID: \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:28:30.867914 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.867891    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-maas-api-tls\") pod \"maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n\" (UID: \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:28:30.956995 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:30.956954    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:31.070508 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:31.070470    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:28:31.408301 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:31.408273    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"]
Sep 10 17:28:31.411385 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:28:31.411349    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f33ebc9_95af_4d29_8d12_eaa8ac61e8b0.slice/crio-26f61d4bd9a1d042c15dba081112ffff8d29999b9c6c076a2810bb08d69405d7 WatchSource:0}: Error finding container 26f61d4bd9a1d042c15dba081112ffff8d29999b9c6c076a2810bb08d69405d7: Status 404 returned error can't find the container with id 26f61d4bd9a1d042c15dba081112ffff8d29999b9c6c076a2810bb08d69405d7
Sep 10 17:28:31.760483 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:31.760451    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n" event={"ID":"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0","Type":"ContainerStarted","Data":"26f61d4bd9a1d042c15dba081112ffff8d29999b9c6c076a2810bb08d69405d7"}
Sep 10 17:28:31.820858 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:31.820771    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:31.831620 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:31.831587    2566 patch_prober.go:28] interesting pod/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 10 17:28:31.831800 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:31.831654    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw" podUID="e0bd13d0-885d-4044-8425-6fe48e338199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 10 17:28:32.668509 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:32.668472    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:32.768705 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:32.768661    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n" event={"ID":"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0","Type":"ContainerStarted","Data":"3ff06895d3b605db6d4c403cb5626420ed519301217d01c760c3938c218ac411"}
Sep 10 17:28:32.768878 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:32.768788    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:28:32.792828 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:32.792777    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n" podStartSLOduration=2.489431304 podStartE2EDuration="2.792763635s" totalImagesPullingTime="303.332331ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:28:32.78886761 +0000 UTC m=+916.470040570" watchObservedRunningTime="2026-09-10 17:28:32.792763635 +0000 UTC m=+916.473936578"
Sep 10 17:28:32.828220 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:32.828191    2566 patch_prober.go:28] interesting pod/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 10 17:28:32.828457 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:32.828431    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw" podUID="e0bd13d0-885d-4044-8425-6fe48e338199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 10 17:28:33.828788 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:33.828741    2566 patch_prober.go:28] interesting pod/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 10 17:28:33.829308 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:33.828813    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw" podUID="e0bd13d0-885d-4044-8425-6fe48e338199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 10 17:28:34.146707 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:34.146611    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:34.900386 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:34.900352    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:34.900904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:34.900727    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:34.902021 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:34.901994    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:28:38.781328 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:38.781296    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:28:49.886321 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:49.886289    2566 generic.go:356] "Generic (PLEG): container finished" podID="52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" containerID="71fa5bfcdb01f2ff68b94daddfd5aa4b7a437c86e4f8ba1e212c6205f3e9da61" exitCode=0
Sep 10 17:28:49.886715 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:49.886366    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh" event={"ID":"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d","Type":"ContainerDied","Data":"71fa5bfcdb01f2ff68b94daddfd5aa4b7a437c86e4f8ba1e212c6205f3e9da61"}
Sep 10 17:28:49.886758 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:49.886715    2566 scope.go:117] "RemoveContainer" containerID="71fa5bfcdb01f2ff68b94daddfd5aa4b7a437c86e4f8ba1e212c6205f3e9da61"
Sep 10 17:28:49.887842 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:49.887811    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:28:49.887954 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:49.887878    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:28:49.887954 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:49.887909    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:28:50.894179 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:50.894115    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh" event={"ID":"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d","Type":"ContainerStarted","Data":"7e5b05260a9da375a173d1e58ee4592ba38d30d4501fe313059170a2fa1a2c56"}
Sep 10 17:28:51.439047 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:51.439011    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:51.656074 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:51.656042    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:51.656268 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:51.656177    2566 patch_prober.go:28] interesting pod/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 10 17:28:51.656268 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:51.656224    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh" podUID="52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 10 17:28:52.656243 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:52.656202    2566 patch_prober.go:28] interesting pod/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 10 17:28:52.656753 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:52.656329    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh" podUID="52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 10 17:28:53.632681 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:53.632632    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:53.656368 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:53.656339    2566 patch_prober.go:28] interesting pod/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 10 17:28:53.656799 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:53.656773    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh" podUID="52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 10 17:28:54.067703 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:54.067654    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:54.162057 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:54.162024    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:54.271622 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:54.271592    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:54.655774 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:54.655744    2566 patch_prober.go:28] interesting pod/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 10 17:28:54.655958 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:54.655805    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh" podUID="52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 10 17:28:54.816947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:54.816913    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:55.656700 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:55.656655    2566 patch_prober.go:28] interesting pod/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 10 17:28:55.656883 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:55.656722    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh" podUID="52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 10 17:28:56.071356 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:56.071308    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:56.660200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:56.660163    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:56.660479 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:56.660457    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:56.661089 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:56.661070    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:28:57.075463 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:57.075421    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:58.082263 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:58.082228    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:58.653778 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:58.653744    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:28:59.133480 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:28:59.133441    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:00.125285 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:00.125246    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:01.078376 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:01.078347    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:01.854466 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:01.854430    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:02.084843 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:02.084804    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:02.372349 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:02.372318    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:02.568887 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:02.568855    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:08.587356 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:08.587315    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:08.773074 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:08.773040    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:08.888280 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:08.888198    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:09.527781 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:09.527750    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"]
Sep 10 17:29:09.528013 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:09.527988    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n" podUID="6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0" containerName="maas-api" containerID="cri-o://3ff06895d3b605db6d4c403cb5626420ed519301217d01c760c3938c218ac411" gracePeriod=30
Sep 10 17:29:09.611310 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:09.611275    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:09.787291 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:09.787261    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:09.802914 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:09.802889    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:29:09.872029 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:09.871995    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jpdqh\" (UniqueName: \"kubernetes.io/projected/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-kube-api-access-jpdqh\") pod \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\" (UID: \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\") "
Sep 10 17:29:09.872223 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:09.872115    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-metrics-certs\") pod \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\" (UID: \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\") "
Sep 10 17:29:09.872223 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:09.872172    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-maas-api-tls\") pod \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\" (UID: \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\") "
Sep 10 17:29:09.872223 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:09.872196    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-openshift-service-ca\") pod \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\" (UID: \"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0\") "
Sep 10 17:29:09.872648 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:09.872620    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0" (UID: "6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:29:09.874327 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:09.874299    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0" (UID: "6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:29:09.874327 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:09.874306    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-kube-api-access-jpdqh" (OuterVolumeSpecName: "kube-api-access-jpdqh") pod "6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0" (UID: "6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0"). InnerVolumeSpecName "kube-api-access-jpdqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:29:09.874684 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:09.874663    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0" (UID: "6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:29:09.973836 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:09.973806    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpdqh\" (UniqueName: \"kubernetes.io/projected/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-kube-api-access-jpdqh\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:29:09.974202 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:09.974181    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-metrics-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:29:09.974440 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:09.974422    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-maas-api-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:29:09.974561 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:09.974547    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0-openshift-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:29:09.977267 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:09.977238    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:10.024659 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:10.024624    2566 generic.go:356] "Generic (PLEG): container finished" podID="6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0" containerID="3ff06895d3b605db6d4c403cb5626420ed519301217d01c760c3938c218ac411" exitCode=0
Sep 10 17:29:10.024842 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:10.024679    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n" event={"ID":"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0","Type":"ContainerDied","Data":"3ff06895d3b605db6d4c403cb5626420ed519301217d01c760c3938c218ac411"}
Sep 10 17:29:10.024842 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:10.024688    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"
Sep 10 17:29:10.024842 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:10.024734    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n" event={"ID":"6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0","Type":"ContainerDied","Data":"26f61d4bd9a1d042c15dba081112ffff8d29999b9c6c076a2810bb08d69405d7"}
Sep 10 17:29:10.024842 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:10.024787    2566 scope.go:117] "RemoveContainer" containerID="3ff06895d3b605db6d4c403cb5626420ed519301217d01c760c3938c218ac411"
Sep 10 17:29:10.037058 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:10.037036    2566 scope.go:117] "RemoveContainer" containerID="3ff06895d3b605db6d4c403cb5626420ed519301217d01c760c3938c218ac411"
Sep 10 17:29:10.037566 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:29:10.037525    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ff06895d3b605db6d4c403cb5626420ed519301217d01c760c3938c218ac411\": container with ID starting with 3ff06895d3b605db6d4c403cb5626420ed519301217d01c760c3938c218ac411 not found: ID does not exist" containerID="3ff06895d3b605db6d4c403cb5626420ed519301217d01c760c3938c218ac411"
Sep 10 17:29:10.037674 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:10.037564    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ff06895d3b605db6d4c403cb5626420ed519301217d01c760c3938c218ac411"} err="failed to get container status \"3ff06895d3b605db6d4c403cb5626420ed519301217d01c760c3938c218ac411\": rpc error: code = NotFound desc = could not find container \"3ff06895d3b605db6d4c403cb5626420ed519301217d01c760c3938c218ac411\": container with ID starting with 3ff06895d3b605db6d4c403cb5626420ed519301217d01c760c3938c218ac411 not found: ID does not exist"
Sep 10 17:29:10.060419 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:10.054767    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"]
Sep 10 17:29:10.065258 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:10.065239    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2d96ccec-664c48985d-gzx9n"]
Sep 10 17:29:10.086560 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:10.086532    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:10.213059 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:10.213005    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:10.326566 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:10.326487    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"]
Sep 10 17:29:10.326790 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:10.326764    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh" podUID="52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" containerName="istio-proxy" containerID="cri-o://7e5b05260a9da375a173d1e58ee4592ba38d30d4501fe313059170a2fa1a2c56" gracePeriod=30
Sep 10 17:29:10.383451 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:10.383409    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:10.734687 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:10.734595    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:10.835173 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:10.835139    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0" path="/var/lib/kubelet/pods/6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0/volumes"
Sep 10 17:29:11.662749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:11.662716    2566 patch_prober.go:28] interesting pod/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 17:29:11.662910 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:11.662782    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh" podUID="52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 17:29:12.006563 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:12.006523    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:12.974787 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:12.974750    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:13.884426 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:13.884396    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:14.068861 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:14.068825    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:14.164193 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:14.164078    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:14.875924 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:14.875887    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:15.580993 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.580970    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:29:15.740132 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.740103    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-token\") pod \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") "
Sep 10 17:29:15.740291 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.740172    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-workload-certs\") pod \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") "
Sep 10 17:29:15.740291 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.740209    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istiod-ca-cert\") pod \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") "
Sep 10 17:29:15.740291 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.740253    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dvbb\" (UniqueName: \"kubernetes.io/projected/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-kube-api-access-7dvbb\") pod \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") "
Sep 10 17:29:15.740291 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.740287    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-data\") pod \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") "
Sep 10 17:29:15.740469 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.740351    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-podinfo\") pod \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") "
Sep 10 17:29:15.740469 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.740409    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-credential-socket\") pod \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") "
Sep 10 17:29:15.740469 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.740446    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-workload-socket\") pod \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") "
Sep 10 17:29:15.740636 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.740469    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-envoy\") pod \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\" (UID: \"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d\") "
Sep 10 17:29:15.740636 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.740535    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" (UID: "52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:29:15.740738 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.740637    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-data" (OuterVolumeSpecName: "istio-data") pod "52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" (UID: "52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:29:15.740738 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.740682    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" (UID: "52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:29:15.740738 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.740716    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" (UID: "52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:29:15.740938 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.740918    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" (UID: "52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:29:15.741323 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.741295    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-credential-socket\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:29:15.741470 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.741327    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-workload-socket\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:29:15.741470 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.741343    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-workload-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:29:15.741470 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.741358    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istiod-ca-cert\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:29:15.741470 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.741371    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-data\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:29:15.742692 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.742661    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-token" (OuterVolumeSpecName: "istio-token") pod "52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" (UID: "52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:29:15.742787 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.742696    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" (UID: "52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:29:15.742787 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.742717    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-kube-api-access-7dvbb" (OuterVolumeSpecName: "kube-api-access-7dvbb") pod "52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" (UID: "52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d"). InnerVolumeSpecName "kube-api-access-7dvbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:29:15.743233 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.743216    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" (UID: "52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 17:29:15.842059 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.842031    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-envoy\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:29:15.842059 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.842057    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-token\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:29:15.842302 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.842066    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dvbb\" (UniqueName: \"kubernetes.io/projected/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-kube-api-access-7dvbb\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:29:15.842302 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.842077    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d-istio-podinfo\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:29:15.944848 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:15.944805    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:16.065444 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:16.065362    2566 generic.go:356] "Generic (PLEG): container finished" podID="52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" containerID="7e5b05260a9da375a173d1e58ee4592ba38d30d4501fe313059170a2fa1a2c56" exitCode=0
Sep 10 17:29:16.065444 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:16.065432    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"
Sep 10 17:29:16.065636 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:16.065445    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh" event={"ID":"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d","Type":"ContainerDied","Data":"7e5b05260a9da375a173d1e58ee4592ba38d30d4501fe313059170a2fa1a2c56"}
Sep 10 17:29:16.065636 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:16.065505    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh" event={"ID":"52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d","Type":"ContainerDied","Data":"bcb53144a938d930e551a88d9bde754e595f4ab7bd0def2cc0f6ff1db7cd9eec"}
Sep 10 17:29:16.065636 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:16.065528    2566 scope.go:117] "RemoveContainer" containerID="7e5b05260a9da375a173d1e58ee4592ba38d30d4501fe313059170a2fa1a2c56"
Sep 10 17:29:16.075831 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:16.075810    2566 scope.go:117] "RemoveContainer" containerID="71fa5bfcdb01f2ff68b94daddfd5aa4b7a437c86e4f8ba1e212c6205f3e9da61"
Sep 10 17:29:16.084949 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:16.084924    2566 scope.go:117] "RemoveContainer" containerID="7e5b05260a9da375a173d1e58ee4592ba38d30d4501fe313059170a2fa1a2c56"
Sep 10 17:29:16.085347 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:29:16.085322    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e5b05260a9da375a173d1e58ee4592ba38d30d4501fe313059170a2fa1a2c56\": container with ID starting with 7e5b05260a9da375a173d1e58ee4592ba38d30d4501fe313059170a2fa1a2c56 not found: ID does not exist" containerID="7e5b05260a9da375a173d1e58ee4592ba38d30d4501fe313059170a2fa1a2c56"
Sep 10 17:29:16.085485 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:16.085360    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e5b05260a9da375a173d1e58ee4592ba38d30d4501fe313059170a2fa1a2c56"} err="failed to get container status \"7e5b05260a9da375a173d1e58ee4592ba38d30d4501fe313059170a2fa1a2c56\": rpc error: code = NotFound desc = could not find container \"7e5b05260a9da375a173d1e58ee4592ba38d30d4501fe313059170a2fa1a2c56\": container with ID starting with 7e5b05260a9da375a173d1e58ee4592ba38d30d4501fe313059170a2fa1a2c56 not found: ID does not exist"
Sep 10 17:29:16.085485 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:16.085385    2566 scope.go:117] "RemoveContainer" containerID="71fa5bfcdb01f2ff68b94daddfd5aa4b7a437c86e4f8ba1e212c6205f3e9da61"
Sep 10 17:29:16.085705 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:29:16.085682    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71fa5bfcdb01f2ff68b94daddfd5aa4b7a437c86e4f8ba1e212c6205f3e9da61\": container with ID starting with 71fa5bfcdb01f2ff68b94daddfd5aa4b7a437c86e4f8ba1e212c6205f3e9da61 not found: ID does not exist" containerID="71fa5bfcdb01f2ff68b94daddfd5aa4b7a437c86e4f8ba1e212c6205f3e9da61"
Sep 10 17:29:16.085801 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:16.085708    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71fa5bfcdb01f2ff68b94daddfd5aa4b7a437c86e4f8ba1e212c6205f3e9da61"} err="failed to get container status \"71fa5bfcdb01f2ff68b94daddfd5aa4b7a437c86e4f8ba1e212c6205f3e9da61\": rpc error: code = NotFound desc = could not find container \"71fa5bfcdb01f2ff68b94daddfd5aa4b7a437c86e4f8ba1e212c6205f3e9da61\": container with ID starting with 71fa5bfcdb01f2ff68b94daddfd5aa4b7a437c86e4f8ba1e212c6205f3e9da61 not found: ID does not exist"
Sep 10 17:29:16.088599 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:16.088579    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"]
Sep 10 17:29:16.097589 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:16.097562    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-2d96ccec-gw-openshift-default-7f9d79dcd8-b4pmh"]
Sep 10 17:29:16.836762 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:16.836724    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" path="/var/lib/kubelet/pods/52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d/volumes"
Sep 10 17:29:16.892724 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:16.892696    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:17.786609 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.786580    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"]
Sep 10 17:29:17.787252 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.787230    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" containerName="istio-proxy"
Sep 10 17:29:17.787252 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.787252    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" containerName="istio-proxy"
Sep 10 17:29:17.787445 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.787275    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" containerName="istio-proxy"
Sep 10 17:29:17.787445 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.787283    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" containerName="istio-proxy"
Sep 10 17:29:17.787445 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.787309    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0" containerName="maas-api"
Sep 10 17:29:17.787445 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.787317    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0" containerName="maas-api"
Sep 10 17:29:17.787445 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.787421    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" containerName="istio-proxy"
Sep 10 17:29:17.787445 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.787435    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f33ebc9-95af-4d29-8d12-eaa8ac61e8b0" containerName="maas-api"
Sep 10 17:29:17.792730 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.792708    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:17.795214 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.795192    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e70241a1-serving-cert\""
Sep 10 17:29:17.801766 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.801745    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"]
Sep 10 17:29:17.829119 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.829087    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:17.866347 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.862465    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84208215-5381-401f-9af6-76ce8abb0ddb-maas-api-tls\") pod \"maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s\" (UID: \"84208215-5381-401f-9af6-76ce8abb0ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:17.866347 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.862546    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84208215-5381-401f-9af6-76ce8abb0ddb-openshift-service-ca\") pod \"maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s\" (UID: \"84208215-5381-401f-9af6-76ce8abb0ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:17.866347 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.862581    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84208215-5381-401f-9af6-76ce8abb0ddb-metrics-certs\") pod \"maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s\" (UID: \"84208215-5381-401f-9af6-76ce8abb0ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:17.866347 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.862643    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbgxq\" (UniqueName: \"kubernetes.io/projected/84208215-5381-401f-9af6-76ce8abb0ddb-kube-api-access-lbgxq\") pod \"maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s\" (UID: \"84208215-5381-401f-9af6-76ce8abb0ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:17.915588 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.915551    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:17.963299 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.963269    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84208215-5381-401f-9af6-76ce8abb0ddb-maas-api-tls\") pod \"maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s\" (UID: \"84208215-5381-401f-9af6-76ce8abb0ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:17.963434 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.963310    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84208215-5381-401f-9af6-76ce8abb0ddb-openshift-service-ca\") pod \"maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s\" (UID: \"84208215-5381-401f-9af6-76ce8abb0ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:17.963434 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.963333    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84208215-5381-401f-9af6-76ce8abb0ddb-metrics-certs\") pod \"maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s\" (UID: \"84208215-5381-401f-9af6-76ce8abb0ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:17.963434 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.963365    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbgxq\" (UniqueName: \"kubernetes.io/projected/84208215-5381-401f-9af6-76ce8abb0ddb-kube-api-access-lbgxq\") pod \"maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s\" (UID: \"84208215-5381-401f-9af6-76ce8abb0ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:17.963434 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:29:17.963423    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-serving-cert: secret "maas-api-e2e-ait-e70241a1-serving-cert" not found
Sep 10 17:29:17.963614 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:29:17.963498    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84208215-5381-401f-9af6-76ce8abb0ddb-maas-api-tls podName:84208215-5381-401f-9af6-76ce8abb0ddb nodeName:}" failed. No retries permitted until 2026-09-10 17:29:18.463477723 +0000 UTC m=+962.144650649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/84208215-5381-401f-9af6-76ce8abb0ddb-maas-api-tls") pod "maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s" (UID: "84208215-5381-401f-9af6-76ce8abb0ddb") : secret "maas-api-e2e-ait-e70241a1-serving-cert" not found
Sep 10 17:29:17.963959 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.963929    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84208215-5381-401f-9af6-76ce8abb0ddb-openshift-service-ca\") pod \"maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s\" (UID: \"84208215-5381-401f-9af6-76ce8abb0ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:17.965807 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.965779    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84208215-5381-401f-9af6-76ce8abb0ddb-metrics-certs\") pod \"maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s\" (UID: \"84208215-5381-401f-9af6-76ce8abb0ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:17.974156 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:17.974118    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbgxq\" (UniqueName: \"kubernetes.io/projected/84208215-5381-401f-9af6-76ce8abb0ddb-kube-api-access-lbgxq\") pod \"maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s\" (UID: \"84208215-5381-401f-9af6-76ce8abb0ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:18.042343 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:18.042268    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:18.439802 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:18.439710    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:18.468419 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:18.468389    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84208215-5381-401f-9af6-76ce8abb0ddb-maas-api-tls\") pod \"maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s\" (UID: \"84208215-5381-401f-9af6-76ce8abb0ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:18.470706 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:18.470678    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84208215-5381-401f-9af6-76ce8abb0ddb-maas-api-tls\") pod \"maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s\" (UID: \"84208215-5381-401f-9af6-76ce8abb0ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:18.535694 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:18.535664    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:18.706585 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:18.706502    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:18.983504 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:18.983333    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"]
Sep 10 17:29:18.986407 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:29:18.986366    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84208215_5381_401f_9af6_76ce8abb0ddb.slice/crio-5c78ee633f7cb0413ea305aec299f9fb3433e461a3f1bc28017c91f53b45d8a4 WatchSource:0}: Error finding container 5c78ee633f7cb0413ea305aec299f9fb3433e461a3f1bc28017c91f53b45d8a4: Status 404 returned error can't find the container with id 5c78ee633f7cb0413ea305aec299f9fb3433e461a3f1bc28017c91f53b45d8a4
Sep 10 17:29:19.086071 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:19.086035    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s" event={"ID":"84208215-5381-401f-9af6-76ce8abb0ddb","Type":"ContainerStarted","Data":"5c78ee633f7cb0413ea305aec299f9fb3433e461a3f1bc28017c91f53b45d8a4"}
Sep 10 17:29:20.094034 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:20.093992    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s" event={"ID":"84208215-5381-401f-9af6-76ce8abb0ddb","Type":"ContainerStarted","Data":"fbcce3c2f259c98b0e68ed6eb156b9bc9fee01408488dda57759ea6ead24ce0e"}
Sep 10 17:29:20.094461 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:20.094139    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:20.117617 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:20.117572    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s" podStartSLOduration=2.798072554 podStartE2EDuration="3.117558542s" totalImagesPullingTime="319.485988ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:29:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:29:20.115165555 +0000 UTC m=+963.796338501" watchObservedRunningTime="2026-09-10 17:29:20.117558542 +0000 UTC m=+963.798731513"
Sep 10 17:29:22.190931 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:22.190887    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:22.291629 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:22.291596    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:26.105371 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:26.105336    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:37.440764 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:37.440725    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:40.615246 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:40.615163    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:41.565014 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:41.564978    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:42.578990 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:42.578940    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:43.361621 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:43.361589    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:44.349246 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:44.349207    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:45.552922 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:45.552887    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:45.690012 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:45.689974    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:46.667988 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:46.667955    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:47.765255 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:47.765222    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:48.695037 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:48.694998    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:49.663472 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:49.663432    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:53.626735 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:53.626700    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:56.238683 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.238648    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"]
Sep 10 17:29:56.239159 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.238891    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s" podUID="84208215-5381-401f-9af6-76ce8abb0ddb" containerName="maas-api" containerID="cri-o://fbcce3c2f259c98b0e68ed6eb156b9bc9fee01408488dda57759ea6ead24ce0e" gracePeriod=30
Sep 10 17:29:56.284161 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.284111    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:56.387012 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.386925    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:56.485097 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.485066    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:56.495092 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.495070    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:56.561931 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.561899    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:56.627191 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.627162    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84208215-5381-401f-9af6-76ce8abb0ddb-openshift-service-ca\") pod \"84208215-5381-401f-9af6-76ce8abb0ddb\" (UID: \"84208215-5381-401f-9af6-76ce8abb0ddb\") "
Sep 10 17:29:56.627352 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.627219    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbgxq\" (UniqueName: \"kubernetes.io/projected/84208215-5381-401f-9af6-76ce8abb0ddb-kube-api-access-lbgxq\") pod \"84208215-5381-401f-9af6-76ce8abb0ddb\" (UID: \"84208215-5381-401f-9af6-76ce8abb0ddb\") "
Sep 10 17:29:56.627352 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.627284    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84208215-5381-401f-9af6-76ce8abb0ddb-metrics-certs\") pod \"84208215-5381-401f-9af6-76ce8abb0ddb\" (UID: \"84208215-5381-401f-9af6-76ce8abb0ddb\") "
Sep 10 17:29:56.627352 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.627313    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84208215-5381-401f-9af6-76ce8abb0ddb-maas-api-tls\") pod \"84208215-5381-401f-9af6-76ce8abb0ddb\" (UID: \"84208215-5381-401f-9af6-76ce8abb0ddb\") "
Sep 10 17:29:56.627599 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.627547    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84208215-5381-401f-9af6-76ce8abb0ddb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "84208215-5381-401f-9af6-76ce8abb0ddb" (UID: "84208215-5381-401f-9af6-76ce8abb0ddb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:29:56.627774 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.627730    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84208215-5381-401f-9af6-76ce8abb0ddb-openshift-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:29:56.629649 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.629621    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84208215-5381-401f-9af6-76ce8abb0ddb-kube-api-access-lbgxq" (OuterVolumeSpecName: "kube-api-access-lbgxq") pod "84208215-5381-401f-9af6-76ce8abb0ddb" (UID: "84208215-5381-401f-9af6-76ce8abb0ddb"). InnerVolumeSpecName "kube-api-access-lbgxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:29:56.629753 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.629665    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84208215-5381-401f-9af6-76ce8abb0ddb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "84208215-5381-401f-9af6-76ce8abb0ddb" (UID: "84208215-5381-401f-9af6-76ce8abb0ddb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:29:56.629875 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.629857    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84208215-5381-401f-9af6-76ce8abb0ddb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "84208215-5381-401f-9af6-76ce8abb0ddb" (UID: "84208215-5381-401f-9af6-76ce8abb0ddb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:29:56.729092 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.729063    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbgxq\" (UniqueName: \"kubernetes.io/projected/84208215-5381-401f-9af6-76ce8abb0ddb-kube-api-access-lbgxq\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:29:56.729092 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.729087    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84208215-5381-401f-9af6-76ce8abb0ddb-metrics-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:29:56.729092 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:56.729097    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84208215-5381-401f-9af6-76ce8abb0ddb-maas-api-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:29:57.008955 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:57.008919    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:57.320560 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:57.320527    2566 generic.go:356] "Generic (PLEG): container finished" podID="84208215-5381-401f-9af6-76ce8abb0ddb" containerID="fbcce3c2f259c98b0e68ed6eb156b9bc9fee01408488dda57759ea6ead24ce0e" exitCode=0
Sep 10 17:29:57.321004 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:57.320569    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"
Sep 10 17:29:57.321004 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:57.320622    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s" event={"ID":"84208215-5381-401f-9af6-76ce8abb0ddb","Type":"ContainerDied","Data":"fbcce3c2f259c98b0e68ed6eb156b9bc9fee01408488dda57759ea6ead24ce0e"}
Sep 10 17:29:57.321004 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:57.320683    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s" event={"ID":"84208215-5381-401f-9af6-76ce8abb0ddb","Type":"ContainerDied","Data":"5c78ee633f7cb0413ea305aec299f9fb3433e461a3f1bc28017c91f53b45d8a4"}
Sep 10 17:29:57.321004 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:57.320713    2566 scope.go:117] "RemoveContainer" containerID="fbcce3c2f259c98b0e68ed6eb156b9bc9fee01408488dda57759ea6ead24ce0e"
Sep 10 17:29:57.330007 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:57.329991    2566 scope.go:117] "RemoveContainer" containerID="fbcce3c2f259c98b0e68ed6eb156b9bc9fee01408488dda57759ea6ead24ce0e"
Sep 10 17:29:57.330290 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:29:57.330271    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbcce3c2f259c98b0e68ed6eb156b9bc9fee01408488dda57759ea6ead24ce0e\": container with ID starting with fbcce3c2f259c98b0e68ed6eb156b9bc9fee01408488dda57759ea6ead24ce0e not found: ID does not exist" containerID="fbcce3c2f259c98b0e68ed6eb156b9bc9fee01408488dda57759ea6ead24ce0e"
Sep 10 17:29:57.330367 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:57.330303    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbcce3c2f259c98b0e68ed6eb156b9bc9fee01408488dda57759ea6ead24ce0e"} err="failed to get container status \"fbcce3c2f259c98b0e68ed6eb156b9bc9fee01408488dda57759ea6ead24ce0e\": rpc error: code = NotFound desc = could not find container \"fbcce3c2f259c98b0e68ed6eb156b9bc9fee01408488dda57759ea6ead24ce0e\": container with ID starting with fbcce3c2f259c98b0e68ed6eb156b9bc9fee01408488dda57759ea6ead24ce0e not found: ID does not exist"
Sep 10 17:29:57.344485 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:57.344457    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"]
Sep 10 17:29:57.355360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:57.355340    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e70241a1-6d4f56d6c5-xnp4s"]
Sep 10 17:29:57.746726 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:57.746699    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:58.021900 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.021814    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:58.145551 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.145517    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:58.434918 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.434839    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"]
Sep 10 17:29:58.435400 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.435377    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84208215-5381-401f-9af6-76ce8abb0ddb" containerName="maas-api"
Sep 10 17:29:58.435400 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.435392    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="84208215-5381-401f-9af6-76ce8abb0ddb" containerName="maas-api"
Sep 10 17:29:58.435524 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.435477    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="52eaae0f-c7d1-4458-9a80-2e4e5ef8c07d" containerName="istio-proxy"
Sep 10 17:29:58.435524 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.435483    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="84208215-5381-401f-9af6-76ce8abb0ddb" containerName="maas-api"
Sep 10 17:29:58.440481 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.440459    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.443074 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.443057    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-dcc1cd6c-openshift-default-dockercfg-z8gh4\""
Sep 10 17:29:58.462756 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.462734    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"]
Sep 10 17:29:58.495462 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.495432    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:58.548688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.548098    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-data\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.548688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.548230    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8df0756-9c18-41e3-a44e-10ea01ce89db-istiod-ca-cert\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.548688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.548417    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-workload-socket\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.548688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.548493    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-token\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.548688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.548551    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-credential-socket\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.548688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.548610    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-envoy\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.548688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.548636    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-podinfo\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.548688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.548705    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-workload-certs\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.549450 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.548760    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbrdd\" (UniqueName: \"kubernetes.io/projected/a8df0756-9c18-41e3-a44e-10ea01ce89db-kube-api-access-cbrdd\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.649944 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.649897    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-workload-certs\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.649944 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.649950    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbrdd\" (UniqueName: \"kubernetes.io/projected/a8df0756-9c18-41e3-a44e-10ea01ce89db-kube-api-access-cbrdd\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.650238 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.650018    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-data\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.650238 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.650052    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8df0756-9c18-41e3-a44e-10ea01ce89db-istiod-ca-cert\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.650238 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.650110    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-workload-socket\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.650469 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.650268    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-token\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.650469 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.650331    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-credential-socket\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.650469 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.650380    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-envoy\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.650469 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.650405    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-podinfo\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.650469 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.650409    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-data\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.650732 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.650468    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-workload-socket\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.650732 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.650722    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-credential-socket\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.650874 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.650852    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8df0756-9c18-41e3-a44e-10ea01ce89db-istiod-ca-cert\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.650933 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.650924    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-workload-certs\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.652725 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.652705    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-envoy\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.653056 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.653035    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-podinfo\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.661877 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.661853    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-token\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.662095 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.662068    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbrdd\" (UniqueName: \"kubernetes.io/projected/a8df0756-9c18-41e3-a44e-10ea01ce89db-kube-api-access-cbrdd\") pod \"e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") " pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.751746 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.751716    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:29:58.836358 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:58.836325    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84208215-5381-401f-9af6-76ce8abb0ddb" path="/var/lib/kubelet/pods/84208215-5381-401f-9af6-76ce8abb0ddb/volumes"
Sep 10 17:29:59.040200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:59.040174    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"]
Sep 10 17:29:59.043267 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:29:59.043239    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8df0756_9c18_41e3_a44e_10ea01ce89db.slice/crio-81dd135874c44394b56d625b5e3a54a84c57ebed9bab2a2dd605abc62181eda3 WatchSource:0}: Error finding container 81dd135874c44394b56d625b5e3a54a84c57ebed9bab2a2dd605abc62181eda3: Status 404 returned error can't find the container with id 81dd135874c44394b56d625b5e3a54a84c57ebed9bab2a2dd605abc62181eda3
Sep 10 17:29:59.046018 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:59.045988    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:59.046199 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:59.046167    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:29:59.046267 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:59.046251    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:29:59.046307 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:59.046290    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:29:59.299432 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:59.299329    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:29:59.336712 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:59.336678    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v" event={"ID":"a8df0756-9c18-41e3-a44e-10ea01ce89db","Type":"ContainerStarted","Data":"fbc62a6051fab4c14b13cb918e288f2c0363880d924b15f82e2b7e956d1c9e05"}
Sep 10 17:29:59.336871 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:59.336736    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v" event={"ID":"a8df0756-9c18-41e3-a44e-10ea01ce89db","Type":"ContainerStarted","Data":"81dd135874c44394b56d625b5e3a54a84c57ebed9bab2a2dd605abc62181eda3"}
Sep 10 17:29:59.370869 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:59.370822    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v" podStartSLOduration=1.370806736 podStartE2EDuration="1.370806736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:29:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:29:59.369831938 +0000 UTC m=+1003.051004880" watchObservedRunningTime="2026-09-10 17:29:59.370806736 +0000 UTC m=+1003.051979678"
Sep 10 17:29:59.752112 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:29:59.752087    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:30:00.170161 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:00.170060    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8c46e8-29817690-bht6g"]
Sep 10 17:30:00.174562 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:00.174539    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2da043-29817690-tpb55"]
Sep 10 17:30:00.174699 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:00.174652    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8c46e8-29817690-bht6g"
Sep 10 17:30:00.177440 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:00.177421    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxddn\""
Sep 10 17:30:00.178303 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:00.178284    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2da043-29817690-tpb55"
Sep 10 17:30:00.194670 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:00.194637    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8c46e8-29817690-bht6g"
Sep 10 17:30:00.199710 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:00.198476    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2da043-29817690-tpb55"]
Sep 10 17:30:00.202197 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:00.202166    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8c46e8-29817690-bht6g"]
Sep 10 17:30:00.203260 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:00.203237    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2da043-29817690-tpb55"
Sep 10 17:30:00.216722 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:00.216691    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:00.486946 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:00.486912    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8c46e8-29817690-bht6g"]
Sep 10 17:30:00.488112 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:30:00.488084    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff19c3ea_8ee9_455d_9989_a5a607dc6cc4.slice/crio-a94179b57a0deb17e503a1a4ac3f37cf0e289ca47685560e0b9432ae583fc2ea WatchSource:0}: Error finding container a94179b57a0deb17e503a1a4ac3f37cf0e289ca47685560e0b9432ae583fc2ea: Status 404 returned error can't find the container with id a94179b57a0deb17e503a1a4ac3f37cf0e289ca47685560e0b9432ae583fc2ea
Sep 10 17:30:00.510606 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:30:00.510571    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod664fb2f5_14e8_4266_a702_e75751ff315c.slice/crio-2f2bcd7f6333b8e825d55baad4199c95186ce066954998a5da78f5e0060a2728 WatchSource:0}: Error finding container 2f2bcd7f6333b8e825d55baad4199c95186ce066954998a5da78f5e0060a2728: Status 404 returned error can't find the container with id 2f2bcd7f6333b8e825d55baad4199c95186ce066954998a5da78f5e0060a2728
Sep 10 17:30:00.511610 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:00.511529    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2da043-29817690-tpb55"]
Sep 10 17:30:00.742523 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:00.742493    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:00.757833 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:00.757807    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:30:00.924965 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:00.924916    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:01.317476 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:01.317443    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:01.351461 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:01.351423    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8c46e8-29817690-bht6g" event={"ID":"ff19c3ea-8ee9-455d-9989-a5a607dc6cc4","Type":"ContainerStarted","Data":"a94179b57a0deb17e503a1a4ac3f37cf0e289ca47685560e0b9432ae583fc2ea"}
Sep 10 17:30:01.352793 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:01.352769    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2da043-29817690-tpb55" event={"ID":"664fb2f5-14e8-4266-a702-e75751ff315c","Type":"ContainerStarted","Data":"2f2bcd7f6333b8e825d55baad4199c95186ce066954998a5da78f5e0060a2728"}
Sep 10 17:30:01.353116 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:01.353096    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:30:01.354053 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:01.354039    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:30:02.805962 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:02.805862    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:03.374503 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:03.374472    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8c46e8-29817690-bht6g" event={"ID":"ff19c3ea-8ee9-455d-9989-a5a607dc6cc4","Type":"ContainerStarted","Data":"7497b493929485f5741228e80c6c32bea5968c29510af251f4ec1af3ebd49fb7"}
Sep 10 17:30:03.402331 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:03.402231    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8c46e8-29817690-bht6g" podStartSLOduration=0.730661046 podStartE2EDuration="3.402214794s" totalImagesPullingTime="2.671553748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:30:03.400085566 +0000 UTC m=+1007.081258510" watchObservedRunningTime="2026-09-10 17:30:03.402214794 +0000 UTC m=+1007.083387737"
Sep 10 17:30:03.902377 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:03.902333    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:04.391054 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:04.391019    2566 generic.go:356] "Generic (PLEG): container finished" podID="ff19c3ea-8ee9-455d-9989-a5a607dc6cc4" containerID="7497b493929485f5741228e80c6c32bea5968c29510af251f4ec1af3ebd49fb7" exitCode=0
Sep 10 17:30:04.391275 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:04.391071    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8c46e8-29817690-bht6g" event={"ID":"ff19c3ea-8ee9-455d-9989-a5a607dc6cc4","Type":"ContainerDied","Data":"7497b493929485f5741228e80c6c32bea5968c29510af251f4ec1af3ebd49fb7"}
Sep 10 17:30:04.392667 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:04.392645    2566 generic.go:356] "Generic (PLEG): container finished" podID="664fb2f5-14e8-4266-a702-e75751ff315c" containerID="c32a9d13d390168decaec9a7c4c4fb5aea7fe2410ad3c30b737cee9c70bbd48a" exitCode=0
Sep 10 17:30:04.392757 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:04.392706    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2da043-29817690-tpb55" event={"ID":"664fb2f5-14e8-4266-a702-e75751ff315c","Type":"ContainerDied","Data":"c32a9d13d390168decaec9a7c4c4fb5aea7fe2410ad3c30b737cee9c70bbd48a"}
Sep 10 17:30:04.821823 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:04.821788    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"]
Sep 10 17:30:04.826761 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:04.826732    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"
Sep 10 17:30:04.830543 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:04.830515    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-dcc1cd6c-serving-cert\""
Sep 10 17:30:04.844231 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:04.844204    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"]
Sep 10 17:30:04.850890 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:04.850864    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:04.917190 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:04.917154    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a27d6f-48cf-40f5-925f-de78c5922e9b-openshift-service-ca\") pod \"maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr\" (UID: \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"
Sep 10 17:30:04.917516 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:04.917209    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4a27d6f-48cf-40f5-925f-de78c5922e9b-metrics-certs\") pod \"maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr\" (UID: \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"
Sep 10 17:30:04.917516 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:04.917261    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s687\" (UniqueName: \"kubernetes.io/projected/a4a27d6f-48cf-40f5-925f-de78c5922e9b-kube-api-access-2s687\") pod \"maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr\" (UID: \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"
Sep 10 17:30:04.917516 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:04.917410    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4a27d6f-48cf-40f5-925f-de78c5922e9b-maas-api-tls\") pod \"maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr\" (UID: \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"
Sep 10 17:30:04.951909 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:04.951876    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:05.018771 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:05.018736    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a27d6f-48cf-40f5-925f-de78c5922e9b-openshift-service-ca\") pod \"maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr\" (UID: \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"
Sep 10 17:30:05.018771 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:05.018774    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4a27d6f-48cf-40f5-925f-de78c5922e9b-metrics-certs\") pod \"maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr\" (UID: \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"
Sep 10 17:30:05.019021 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:05.018808    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s687\" (UniqueName: \"kubernetes.io/projected/a4a27d6f-48cf-40f5-925f-de78c5922e9b-kube-api-access-2s687\") pod \"maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr\" (UID: \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"
Sep 10 17:30:05.019021 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:05.018894    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4a27d6f-48cf-40f5-925f-de78c5922e9b-maas-api-tls\") pod \"maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr\" (UID: \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"
Sep 10 17:30:05.019559 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:05.019531    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a27d6f-48cf-40f5-925f-de78c5922e9b-openshift-service-ca\") pod \"maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr\" (UID: \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"
Sep 10 17:30:05.021337 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:05.021312    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4a27d6f-48cf-40f5-925f-de78c5922e9b-maas-api-tls\") pod \"maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr\" (UID: \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"
Sep 10 17:30:05.021445 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:05.021344    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4a27d6f-48cf-40f5-925f-de78c5922e9b-metrics-certs\") pod \"maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr\" (UID: \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"
Sep 10 17:30:05.027428 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:05.027404    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s687\" (UniqueName: \"kubernetes.io/projected/a4a27d6f-48cf-40f5-925f-de78c5922e9b-kube-api-access-2s687\") pod \"maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr\" (UID: \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"
Sep 10 17:30:05.052406 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:05.052373    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:05.139344 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:05.139263    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"
Sep 10 17:30:05.440930 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:05.440870    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:05.452354 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:30:05.452317    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a27d6f_48cf_40f5_925f_de78c5922e9b.slice/crio-a5a552503333869280fb00b775e9f79e384ebd455313a0240c5575a97d69d0dd WatchSource:0}: Error finding container a5a552503333869280fb00b775e9f79e384ebd455313a0240c5575a97d69d0dd: Status 404 returned error can't find the container with id a5a552503333869280fb00b775e9f79e384ebd455313a0240c5575a97d69d0dd
Sep 10 17:30:05.452756 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:05.452733    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"]
Sep 10 17:30:05.546841 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:05.546818    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2da043-29817690-tpb55"
Sep 10 17:30:05.579651 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:05.579619    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:05.580816 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:05.580797    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8c46e8-29817690-bht6g"
Sep 10 17:30:06.406236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:06.406211    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8c46e8-29817690-bht6g"
Sep 10 17:30:06.406647 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:06.406212    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8c46e8-29817690-bht6g" event={"ID":"ff19c3ea-8ee9-455d-9989-a5a607dc6cc4","Type":"ContainerDied","Data":"a94179b57a0deb17e503a1a4ac3f37cf0e289ca47685560e0b9432ae583fc2ea"}
Sep 10 17:30:06.406647 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:06.406331    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a94179b57a0deb17e503a1a4ac3f37cf0e289ca47685560e0b9432ae583fc2ea"
Sep 10 17:30:06.407801 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:06.407773    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr" event={"ID":"a4a27d6f-48cf-40f5-925f-de78c5922e9b","Type":"ContainerStarted","Data":"bdb9f4e5341830a207e52234ae24f64f95a718f9863b647393fc51be5ecea6e7"}
Sep 10 17:30:06.407939 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:06.407826    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"
Sep 10 17:30:06.407939 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:06.407852    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr" event={"ID":"a4a27d6f-48cf-40f5-925f-de78c5922e9b","Type":"ContainerStarted","Data":"a5a552503333869280fb00b775e9f79e384ebd455313a0240c5575a97d69d0dd"}
Sep 10 17:30:06.409197 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:06.409174    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2da043-29817690-tpb55" event={"ID":"664fb2f5-14e8-4266-a702-e75751ff315c","Type":"ContainerDied","Data":"2f2bcd7f6333b8e825d55baad4199c95186ce066954998a5da78f5e0060a2728"}
Sep 10 17:30:06.409333 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:06.409214    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2f2bcd7f6333b8e825d55baad4199c95186ce066954998a5da78f5e0060a2728"
Sep 10 17:30:06.409333 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:06.409217    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2da043-29817690-tpb55"
Sep 10 17:30:06.442453 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:06.442406    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr" podStartSLOduration=2.054345075 podStartE2EDuration="2.442392088s" totalImagesPullingTime="388.047013ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:30:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:30:06.439435581 +0000 UTC m=+1010.120608518" watchObservedRunningTime="2026-09-10 17:30:06.442392088 +0000 UTC m=+1010.123565031"
Sep 10 17:30:12.421601 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:12.421569    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"
Sep 10 17:30:17.634758 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:17.634722    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:17.776667 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:17.776634    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:19.936989 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:19.936940    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:20.264509 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:20.264469    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:20.991992 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:20.991957    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:21.441572 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:21.441482    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:22.459766 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:22.459726    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:23.369331 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:23.369296    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:24.532705 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:24.532672    2566 generic.go:356] "Generic (PLEG): container finished" podID="a8df0756-9c18-41e3-a44e-10ea01ce89db" containerID="fbc62a6051fab4c14b13cb918e288f2c0363880d924b15f82e2b7e956d1c9e05" exitCode=0
Sep 10 17:30:24.533206 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:24.532748    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v" event={"ID":"a8df0756-9c18-41e3-a44e-10ea01ce89db","Type":"ContainerDied","Data":"fbc62a6051fab4c14b13cb918e288f2c0363880d924b15f82e2b7e956d1c9e05"}
Sep 10 17:30:24.533206 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:24.533195    2566 scope.go:117] "RemoveContainer" containerID="fbc62a6051fab4c14b13cb918e288f2c0363880d924b15f82e2b7e956d1c9e05"
Sep 10 17:30:24.534277 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:24.534252    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:30:24.534354 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:24.534315    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:30:24.534354 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:24.534346    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:30:24.676251 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:24.676217    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:25.543869 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:25.543836    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v" event={"ID":"a8df0756-9c18-41e3-a44e-10ea01ce89db","Type":"ContainerStarted","Data":"3800715e13c7fa1ce6afef8174805316f7a80b6462b80fe0bb7bce520feff6f4"}
Sep 10 17:30:25.588771 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:25.588716    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:25.752247 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:25.752206    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:30:25.752482 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:25.752457    2566 patch_prober.go:28] interesting pod/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 10 17:30:25.752541 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:25.752524    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v" podUID="a8df0756-9c18-41e3-a44e-10ea01ce89db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 10 17:30:26.577143 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:26.577101    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:26.753154 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:26.753093    2566 patch_prober.go:28] interesting pod/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 10 17:30:26.753357 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:26.753189    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v" podUID="a8df0756-9c18-41e3-a44e-10ea01ce89db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 10 17:30:27.598502 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:27.598463    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:27.752893 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:27.752860    2566 patch_prober.go:28] interesting pod/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 10 17:30:27.753054 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:27.752912    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v" podUID="a8df0756-9c18-41e3-a44e-10ea01ce89db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 10 17:30:28.596143 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:28.596088    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:28.752614 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:28.752579    2566 patch_prober.go:28] interesting pod/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 10 17:30:28.752995 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:28.752644    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v" podUID="a8df0756-9c18-41e3-a44e-10ea01ce89db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 10 17:30:29.431727 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:29.431684    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:29.752097 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:29.752061    2566 patch_prober.go:28] interesting pod/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 10 17:30:29.752307 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:29.752135    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v" podUID="a8df0756-9c18-41e3-a44e-10ea01ce89db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 10 17:30:30.355615 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:30.355578    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:30.947104 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:30.947066    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:30:30.947541 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:30.947502    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:30:30.949198 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:30.949176    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:30:31.498684 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:31.498649    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:32.556756 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:32.556722    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:33.367156 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:33.367111    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:34.089103 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.089061    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"]
Sep 10 17:30:34.089520 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.089316    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs" podUID="2a70068e-8b2f-49f5-9893-80290b22b244" containerName="maas-api" containerID="cri-o://0ce7b1c1e014c89fe4ff18890bcc5e7b717a5576c2b4bdec998b9dbdf05d5dd4" gracePeriod=30
Sep 10 17:30:34.141785 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.141748    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:34.146614 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.146583    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2da043-29817690-tpb55"]
Sep 10 17:30:34.149265 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.149240    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2da043-29817690-tpb55"]
Sep 10 17:30:34.196901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.196864    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf"]
Sep 10 17:30:34.197120 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.197087    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf" podUID="97ba04cb-05e1-418a-8dac-e5b951f997ae" containerName="payload-pre-processing" containerID="cri-o://4d664187abcf3d0fea1bb7240ca1f37c3dd8f352fd65357170a92e24383267bf" gracePeriod=30
Sep 10 17:30:34.260085 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.260043    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:34.357037 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.356598    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:30:34.363286 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.363258    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:34.425385 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.425343    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a70068e-8b2f-49f5-9893-80290b22b244-maas-api-tls\") pod \"2a70068e-8b2f-49f5-9893-80290b22b244\" (UID: \"2a70068e-8b2f-49f5-9893-80290b22b244\") "
Sep 10 17:30:34.425549 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.425415    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a70068e-8b2f-49f5-9893-80290b22b244-metrics-certs\") pod \"2a70068e-8b2f-49f5-9893-80290b22b244\" (UID: \"2a70068e-8b2f-49f5-9893-80290b22b244\") "
Sep 10 17:30:34.425549 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.425441    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a70068e-8b2f-49f5-9893-80290b22b244-openshift-service-ca\") pod \"2a70068e-8b2f-49f5-9893-80290b22b244\" (UID: \"2a70068e-8b2f-49f5-9893-80290b22b244\") "
Sep 10 17:30:34.425549 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.425468    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsqhr\" (UniqueName: \"kubernetes.io/projected/2a70068e-8b2f-49f5-9893-80290b22b244-kube-api-access-fsqhr\") pod \"2a70068e-8b2f-49f5-9893-80290b22b244\" (UID: \"2a70068e-8b2f-49f5-9893-80290b22b244\") "
Sep 10 17:30:34.426110 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.426068    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a70068e-8b2f-49f5-9893-80290b22b244-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2a70068e-8b2f-49f5-9893-80290b22b244" (UID: "2a70068e-8b2f-49f5-9893-80290b22b244"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:30:34.429509 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.429483    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a70068e-8b2f-49f5-9893-80290b22b244-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2a70068e-8b2f-49f5-9893-80290b22b244" (UID: "2a70068e-8b2f-49f5-9893-80290b22b244"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:30:34.430766 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.430734    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a70068e-8b2f-49f5-9893-80290b22b244-kube-api-access-fsqhr" (OuterVolumeSpecName: "kube-api-access-fsqhr") pod "2a70068e-8b2f-49f5-9893-80290b22b244" (UID: "2a70068e-8b2f-49f5-9893-80290b22b244"). InnerVolumeSpecName "kube-api-access-fsqhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:30:34.430858 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.430775    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a70068e-8b2f-49f5-9893-80290b22b244-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2a70068e-8b2f-49f5-9893-80290b22b244" (UID: "2a70068e-8b2f-49f5-9893-80290b22b244"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:30:34.462794 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.462771    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf"
Sep 10 17:30:34.483223 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.483190    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:34.526330 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.526300    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97ba04cb-05e1-418a-8dac-e5b951f997ae-plugins-config-volume\") pod \"97ba04cb-05e1-418a-8dac-e5b951f997ae\" (UID: \"97ba04cb-05e1-418a-8dac-e5b951f997ae\") "
Sep 10 17:30:34.526471 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.526407    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvcvc\" (UniqueName: \"kubernetes.io/projected/97ba04cb-05e1-418a-8dac-e5b951f997ae-kube-api-access-mvcvc\") pod \"97ba04cb-05e1-418a-8dac-e5b951f997ae\" (UID: \"97ba04cb-05e1-418a-8dac-e5b951f997ae\") "
Sep 10 17:30:34.526646 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.526622    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97ba04cb-05e1-418a-8dac-e5b951f997ae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "97ba04cb-05e1-418a-8dac-e5b951f997ae" (UID: "97ba04cb-05e1-418a-8dac-e5b951f997ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:30:34.526716 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.526671    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a70068e-8b2f-49f5-9893-80290b22b244-maas-api-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:34.526716 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.526684    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a70068e-8b2f-49f5-9893-80290b22b244-metrics-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:34.526716 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.526692    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a70068e-8b2f-49f5-9893-80290b22b244-openshift-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:34.526716 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.526701    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsqhr\" (UniqueName: \"kubernetes.io/projected/2a70068e-8b2f-49f5-9893-80290b22b244-kube-api-access-fsqhr\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:34.528383 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.528361    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97ba04cb-05e1-418a-8dac-e5b951f997ae-kube-api-access-mvcvc" (OuterVolumeSpecName: "kube-api-access-mvcvc") pod "97ba04cb-05e1-418a-8dac-e5b951f997ae" (UID: "97ba04cb-05e1-418a-8dac-e5b951f997ae"). InnerVolumeSpecName "kube-api-access-mvcvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:30:34.581792 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.581756    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:34.600616 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.600578    2566 generic.go:356] "Generic (PLEG): container finished" podID="97ba04cb-05e1-418a-8dac-e5b951f997ae" containerID="4d664187abcf3d0fea1bb7240ca1f37c3dd8f352fd65357170a92e24383267bf" exitCode=0
Sep 10 17:30:34.600768 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.600634    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf"
Sep 10 17:30:34.600768 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.600666    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf" event={"ID":"97ba04cb-05e1-418a-8dac-e5b951f997ae","Type":"ContainerDied","Data":"4d664187abcf3d0fea1bb7240ca1f37c3dd8f352fd65357170a92e24383267bf"}
Sep 10 17:30:34.600768 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.600726    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf" event={"ID":"97ba04cb-05e1-418a-8dac-e5b951f997ae","Type":"ContainerDied","Data":"c2449820f23ea0ad8b68e7337d15ff787c6a61ed2df2616bb3c88ee46d4fbc81"}
Sep 10 17:30:34.600768 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.600756    2566 scope.go:117] "RemoveContainer" containerID="4d664187abcf3d0fea1bb7240ca1f37c3dd8f352fd65357170a92e24383267bf"
Sep 10 17:30:34.602323 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.602300    2566 generic.go:356] "Generic (PLEG): container finished" podID="2a70068e-8b2f-49f5-9893-80290b22b244" containerID="0ce7b1c1e014c89fe4ff18890bcc5e7b717a5576c2b4bdec998b9dbdf05d5dd4" exitCode=0
Sep 10 17:30:34.602439 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.602330    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"
Sep 10 17:30:34.602439 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.602354    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs" event={"ID":"2a70068e-8b2f-49f5-9893-80290b22b244","Type":"ContainerDied","Data":"0ce7b1c1e014c89fe4ff18890bcc5e7b717a5576c2b4bdec998b9dbdf05d5dd4"}
Sep 10 17:30:34.602439 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.602397    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs" event={"ID":"2a70068e-8b2f-49f5-9893-80290b22b244","Type":"ContainerDied","Data":"1006aba4a07cc66201b8153639ef1f48cf64b086a1803639bfdf6455a52809bc"}
Sep 10 17:30:34.611681 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.611650    2566 scope.go:117] "RemoveContainer" containerID="4d664187abcf3d0fea1bb7240ca1f37c3dd8f352fd65357170a92e24383267bf"
Sep 10 17:30:34.612005 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:30:34.611985    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d664187abcf3d0fea1bb7240ca1f37c3dd8f352fd65357170a92e24383267bf\": container with ID starting with 4d664187abcf3d0fea1bb7240ca1f37c3dd8f352fd65357170a92e24383267bf not found: ID does not exist" containerID="4d664187abcf3d0fea1bb7240ca1f37c3dd8f352fd65357170a92e24383267bf"
Sep 10 17:30:34.612100 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.612036    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d664187abcf3d0fea1bb7240ca1f37c3dd8f352fd65357170a92e24383267bf"} err="failed to get container status \"4d664187abcf3d0fea1bb7240ca1f37c3dd8f352fd65357170a92e24383267bf\": rpc error: code = NotFound desc = could not find container \"4d664187abcf3d0fea1bb7240ca1f37c3dd8f352fd65357170a92e24383267bf\": container with ID starting with 4d664187abcf3d0fea1bb7240ca1f37c3dd8f352fd65357170a92e24383267bf not found: ID does not exist"
Sep 10 17:30:34.612100 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.612054    2566 scope.go:117] "RemoveContainer" containerID="0ce7b1c1e014c89fe4ff18890bcc5e7b717a5576c2b4bdec998b9dbdf05d5dd4"
Sep 10 17:30:34.620993 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.620976    2566 scope.go:117] "RemoveContainer" containerID="0ce7b1c1e014c89fe4ff18890bcc5e7b717a5576c2b4bdec998b9dbdf05d5dd4"
Sep 10 17:30:34.621300 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:30:34.621281    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ce7b1c1e014c89fe4ff18890bcc5e7b717a5576c2b4bdec998b9dbdf05d5dd4\": container with ID starting with 0ce7b1c1e014c89fe4ff18890bcc5e7b717a5576c2b4bdec998b9dbdf05d5dd4 not found: ID does not exist" containerID="0ce7b1c1e014c89fe4ff18890bcc5e7b717a5576c2b4bdec998b9dbdf05d5dd4"
Sep 10 17:30:34.621376 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.621309    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ce7b1c1e014c89fe4ff18890bcc5e7b717a5576c2b4bdec998b9dbdf05d5dd4"} err="failed to get container status \"0ce7b1c1e014c89fe4ff18890bcc5e7b717a5576c2b4bdec998b9dbdf05d5dd4\": rpc error: code = NotFound desc = could not find container \"0ce7b1c1e014c89fe4ff18890bcc5e7b717a5576c2b4bdec998b9dbdf05d5dd4\": container with ID starting with 0ce7b1c1e014c89fe4ff18890bcc5e7b717a5576c2b4bdec998b9dbdf05d5dd4 not found: ID does not exist"
Sep 10 17:30:34.627454 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.627432    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97ba04cb-05e1-418a-8dac-e5b951f997ae-plugins-config-volume\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:34.627566 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.627457    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvcvc\" (UniqueName: \"kubernetes.io/projected/97ba04cb-05e1-418a-8dac-e5b951f997ae-kube-api-access-mvcvc\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:34.630748 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.630722    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"]
Sep 10 17:30:34.633416 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.633395    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-2da043-5858558c65-b6fvs"]
Sep 10 17:30:34.653244 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.653220    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf"]
Sep 10 17:30:34.662850 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.662829    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2da043-756b789474-npbjf"]
Sep 10 17:30:34.841557 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.841515    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a70068e-8b2f-49f5-9893-80290b22b244" path="/var/lib/kubelet/pods/2a70068e-8b2f-49f5-9893-80290b22b244/volumes"
Sep 10 17:30:34.841907 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.841887    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="664fb2f5-14e8-4266-a702-e75751ff315c" path="/var/lib/kubelet/pods/664fb2f5-14e8-4266-a702-e75751ff315c/volumes"
Sep 10 17:30:34.842289 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.842270    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97ba04cb-05e1-418a-8dac-e5b951f997ae" path="/var/lib/kubelet/pods/97ba04cb-05e1-418a-8dac-e5b951f997ae/volumes"
Sep 10 17:30:34.864671 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:34.864592    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:35.711453 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:35.711419    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:36.624749 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:36.624716    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:37.810730 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:37.810691    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:38.964027 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:38.963992    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:41.208438 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:41.208403    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:41.328854 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:41.328813    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:41.542538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:41.542505    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"]
Sep 10 17:30:41.542723 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:41.542698    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d" podUID="0e80813d-281d-4109-8e31-946428b49733" containerName="istio-proxy" containerID="cri-o://7bc55f1fc6ec0caa8b47350ee809d3c8db24b324be787423ba6d56716de267df" gracePeriod=30
Sep 10 17:30:41.630409 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:41.630364    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:41.986336 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:41.986251    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:42.276531 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:42.276498    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:42.422462 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:42.422348    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:42.704585 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:42.704506    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:43.089690 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.089653    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"]
Sep 10 17:30:43.089935 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.089908    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr" podUID="a4a27d6f-48cf-40f5-925f-de78c5922e9b" containerName="maas-api" containerID="cri-o://bdb9f4e5341830a207e52234ae24f64f95a718f9863b647393fc51be5ecea6e7" gracePeriod=30
Sep 10 17:30:43.123022 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.122986    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:43.228467 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.228432    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:43.324460 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.324430    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:43.346204 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.346155    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"
Sep 10 17:30:43.418116 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.418074    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:43.510290 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.510250    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:43.516343 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.516314    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a27d6f-48cf-40f5-925f-de78c5922e9b-openshift-service-ca\") pod \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\" (UID: \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\") "
Sep 10 17:30:43.516490 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.516365    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4a27d6f-48cf-40f5-925f-de78c5922e9b-maas-api-tls\") pod \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\" (UID: \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\") "
Sep 10 17:30:43.516490 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.516395    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4a27d6f-48cf-40f5-925f-de78c5922e9b-metrics-certs\") pod \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\" (UID: \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\") "
Sep 10 17:30:43.516642 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.516492    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2s687\" (UniqueName: \"kubernetes.io/projected/a4a27d6f-48cf-40f5-925f-de78c5922e9b-kube-api-access-2s687\") pod \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\" (UID: \"a4a27d6f-48cf-40f5-925f-de78c5922e9b\") "
Sep 10 17:30:43.516729 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.516690    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4a27d6f-48cf-40f5-925f-de78c5922e9b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a4a27d6f-48cf-40f5-925f-de78c5922e9b" (UID: "a4a27d6f-48cf-40f5-925f-de78c5922e9b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:30:43.516947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.516926    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a27d6f-48cf-40f5-925f-de78c5922e9b-openshift-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:43.519478 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.519438    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4a27d6f-48cf-40f5-925f-de78c5922e9b-kube-api-access-2s687" (OuterVolumeSpecName: "kube-api-access-2s687") pod "a4a27d6f-48cf-40f5-925f-de78c5922e9b" (UID: "a4a27d6f-48cf-40f5-925f-de78c5922e9b"). InnerVolumeSpecName "kube-api-access-2s687". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:30:43.519594 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.519493    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a27d6f-48cf-40f5-925f-de78c5922e9b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a4a27d6f-48cf-40f5-925f-de78c5922e9b" (UID: "a4a27d6f-48cf-40f5-925f-de78c5922e9b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:30:43.519786 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.519762    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a27d6f-48cf-40f5-925f-de78c5922e9b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a4a27d6f-48cf-40f5-925f-de78c5922e9b" (UID: "a4a27d6f-48cf-40f5-925f-de78c5922e9b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:30:43.601558 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.601478    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"]
Sep 10 17:30:43.601697 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.601670    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v" podUID="a8df0756-9c18-41e3-a44e-10ea01ce89db" containerName="istio-proxy" containerID="cri-o://3800715e13c7fa1ce6afef8174805316f7a80b6462b80fe0bb7bce520feff6f4" gracePeriod=30
Sep 10 17:30:43.617589 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.617559    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2s687\" (UniqueName: \"kubernetes.io/projected/a4a27d6f-48cf-40f5-925f-de78c5922e9b-kube-api-access-2s687\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:43.617589 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.617583    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4a27d6f-48cf-40f5-925f-de78c5922e9b-maas-api-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:43.617589 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.617592    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4a27d6f-48cf-40f5-925f-de78c5922e9b-metrics-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:43.662753 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.662715    2566 generic.go:356] "Generic (PLEG): container finished" podID="a4a27d6f-48cf-40f5-925f-de78c5922e9b" containerID="bdb9f4e5341830a207e52234ae24f64f95a718f9863b647393fc51be5ecea6e7" exitCode=0
Sep 10 17:30:43.662906 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.662889    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr" event={"ID":"a4a27d6f-48cf-40f5-925f-de78c5922e9b","Type":"ContainerDied","Data":"bdb9f4e5341830a207e52234ae24f64f95a718f9863b647393fc51be5ecea6e7"}
Sep 10 17:30:43.663027 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.663010    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr" event={"ID":"a4a27d6f-48cf-40f5-925f-de78c5922e9b","Type":"ContainerDied","Data":"a5a552503333869280fb00b775e9f79e384ebd455313a0240c5575a97d69d0dd"}
Sep 10 17:30:43.663098 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.663086    2566 scope.go:117] "RemoveContainer" containerID="bdb9f4e5341830a207e52234ae24f64f95a718f9863b647393fc51be5ecea6e7"
Sep 10 17:30:43.663340 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.663286    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"
Sep 10 17:30:43.679294 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.679269    2566 scope.go:117] "RemoveContainer" containerID="bdb9f4e5341830a207e52234ae24f64f95a718f9863b647393fc51be5ecea6e7"
Sep 10 17:30:43.679689 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.679653    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:43.679689 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:30:43.679669    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdb9f4e5341830a207e52234ae24f64f95a718f9863b647393fc51be5ecea6e7\": container with ID starting with bdb9f4e5341830a207e52234ae24f64f95a718f9863b647393fc51be5ecea6e7 not found: ID does not exist" containerID="bdb9f4e5341830a207e52234ae24f64f95a718f9863b647393fc51be5ecea6e7"
Sep 10 17:30:43.679860 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.679705    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdb9f4e5341830a207e52234ae24f64f95a718f9863b647393fc51be5ecea6e7"} err="failed to get container status \"bdb9f4e5341830a207e52234ae24f64f95a718f9863b647393fc51be5ecea6e7\": rpc error: code = NotFound desc = could not find container \"bdb9f4e5341830a207e52234ae24f64f95a718f9863b647393fc51be5ecea6e7\": container with ID starting with bdb9f4e5341830a207e52234ae24f64f95a718f9863b647393fc51be5ecea6e7 not found: ID does not exist"
Sep 10 17:30:43.706827 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.706799    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"]
Sep 10 17:30:43.715500 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.715476    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-dcc1cd6c-bdcb8f494-vzcsr"]
Sep 10 17:30:43.974051 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:43.973968    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:44.330076 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:44.330045    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:44.471874 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:44.471839    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:44.835651 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:44.835617    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4a27d6f-48cf-40f5-925f-de78c5922e9b" path="/var/lib/kubelet/pods/a4a27d6f-48cf-40f5-925f-de78c5922e9b/volumes"
Sep 10 17:30:45.346714 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:45.346681    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:45.944094 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:45.944066    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:45.949862 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:45.949831    2566 patch_prober.go:28] interesting pod/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 17:30:45.949995 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:45.949890    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v" podUID="a8df0756-9c18-41e3-a44e-10ea01ce89db" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 17:30:46.539167 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.539120    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:46.685366 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.685332    2566 generic.go:356] "Generic (PLEG): container finished" podID="0e80813d-281d-4109-8e31-946428b49733" containerID="7bc55f1fc6ec0caa8b47350ee809d3c8db24b324be787423ba6d56716de267df" exitCode=0
Sep 10 17:30:46.685499 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.685372    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d" event={"ID":"0e80813d-281d-4109-8e31-946428b49733","Type":"ContainerDied","Data":"7bc55f1fc6ec0caa8b47350ee809d3c8db24b324be787423ba6d56716de267df"}
Sep 10 17:30:46.685499 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.685460    2566 scope.go:117] "RemoveContainer" containerID="235426d15ebeb9831e6240377c3a2b85df25517288f79c1be09de691125bb3d9"
Sep 10 17:30:46.809843 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.809545    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:30:46.959106 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959077    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e80813d-281d-4109-8e31-946428b49733-istiod-ca-cert\") pod \"0e80813d-281d-4109-8e31-946428b49733\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") "
Sep 10 17:30:46.959306 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959159    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-credential-socket\") pod \"0e80813d-281d-4109-8e31-946428b49733\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") "
Sep 10 17:30:46.959306 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959192    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e80813d-281d-4109-8e31-946428b49733-istio-podinfo\") pod \"0e80813d-281d-4109-8e31-946428b49733\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") "
Sep 10 17:30:46.959306 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959212    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-istio-envoy\") pod \"0e80813d-281d-4109-8e31-946428b49733\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") "
Sep 10 17:30:46.959306 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959237    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e80813d-281d-4109-8e31-946428b49733-istio-token\") pod \"0e80813d-281d-4109-8e31-946428b49733\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") "
Sep 10 17:30:46.959306 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959277    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24xf6\" (UniqueName: \"kubernetes.io/projected/0e80813d-281d-4109-8e31-946428b49733-kube-api-access-24xf6\") pod \"0e80813d-281d-4109-8e31-946428b49733\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") "
Sep 10 17:30:46.959571 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959312    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-workload-socket\") pod \"0e80813d-281d-4109-8e31-946428b49733\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") "
Sep 10 17:30:46.959571 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959398    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-istio-data\") pod \"0e80813d-281d-4109-8e31-946428b49733\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") "
Sep 10 17:30:46.959571 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959424    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-workload-certs\") pod \"0e80813d-281d-4109-8e31-946428b49733\" (UID: \"0e80813d-281d-4109-8e31-946428b49733\") "
Sep 10 17:30:46.959571 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959543    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0e80813d-281d-4109-8e31-946428b49733" (UID: "0e80813d-281d-4109-8e31-946428b49733"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:30:46.959777 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959545    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e80813d-281d-4109-8e31-946428b49733-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0e80813d-281d-4109-8e31-946428b49733" (UID: "0e80813d-281d-4109-8e31-946428b49733"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:30:46.959777 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959682    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0e80813d-281d-4109-8e31-946428b49733" (UID: "0e80813d-281d-4109-8e31-946428b49733"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:30:46.959777 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959746    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-istio-data" (OuterVolumeSpecName: "istio-data") pod "0e80813d-281d-4109-8e31-946428b49733" (UID: "0e80813d-281d-4109-8e31-946428b49733"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:30:46.959777 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959765    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0e80813d-281d-4109-8e31-946428b49733" (UID: "0e80813d-281d-4109-8e31-946428b49733"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:30:46.959971 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959908    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e80813d-281d-4109-8e31-946428b49733-istiod-ca-cert\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:46.959971 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959927    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-credential-socket\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:46.959971 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959942    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-workload-socket\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:46.959971 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959954    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-istio-data\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:46.959971 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.959967    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-workload-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:46.961538 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.961510    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0e80813d-281d-4109-8e31-946428b49733" (UID: "0e80813d-281d-4109-8e31-946428b49733"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:30:46.961649 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.961603    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e80813d-281d-4109-8e31-946428b49733-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0e80813d-281d-4109-8e31-946428b49733" (UID: "0e80813d-281d-4109-8e31-946428b49733"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 17:30:46.961795 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.961777    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e80813d-281d-4109-8e31-946428b49733-istio-token" (OuterVolumeSpecName: "istio-token") pod "0e80813d-281d-4109-8e31-946428b49733" (UID: "0e80813d-281d-4109-8e31-946428b49733"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:30:46.961842 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:46.961803    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e80813d-281d-4109-8e31-946428b49733-kube-api-access-24xf6" (OuterVolumeSpecName: "kube-api-access-24xf6") pod "0e80813d-281d-4109-8e31-946428b49733" (UID: "0e80813d-281d-4109-8e31-946428b49733"). InnerVolumeSpecName "kube-api-access-24xf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:30:47.061348 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:47.061263    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e80813d-281d-4109-8e31-946428b49733-istio-podinfo\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:47.061348 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:47.061290    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e80813d-281d-4109-8e31-946428b49733-istio-envoy\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:47.061348 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:47.061301    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e80813d-281d-4109-8e31-946428b49733-istio-token\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:47.061348 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:47.061316    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24xf6\" (UniqueName: \"kubernetes.io/projected/0e80813d-281d-4109-8e31-946428b49733-kube-api-access-24xf6\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:47.692967 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:47.692936    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"
Sep 10 17:30:47.693394 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:47.692975    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d" event={"ID":"0e80813d-281d-4109-8e31-946428b49733","Type":"ContainerDied","Data":"ffcd91b10796fb820d6d6c292dcac97a91185e67709aa25a34f815e032471a3b"}
Sep 10 17:30:47.693394 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:47.693035    2566 scope.go:117] "RemoveContainer" containerID="7bc55f1fc6ec0caa8b47350ee809d3c8db24b324be787423ba6d56716de267df"
Sep 10 17:30:47.736697 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:47.736666    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"]
Sep 10 17:30:47.748566 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:47.748540    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-2da043-openshift-default-77ddd5f5d6-59p2d"]
Sep 10 17:30:47.751935 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:47.751911    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:47.941285 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:47.941236    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:48.064037 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.063999    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:48.795712 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.795671    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:48.835295 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.835259    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e80813d-281d-4109-8e31-946428b49733" path="/var/lib/kubelet/pods/0e80813d-281d-4109-8e31-946428b49733/volumes"
Sep 10 17:30:48.858365 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.858342    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:30:48.877135 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877064    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-data\") pod \"a8df0756-9c18-41e3-a44e-10ea01ce89db\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") "
Sep 10 17:30:48.877247 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877162    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-workload-certs\") pod \"a8df0756-9c18-41e3-a44e-10ea01ce89db\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") "
Sep 10 17:30:48.877247 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877218    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-workload-socket\") pod \"a8df0756-9c18-41e3-a44e-10ea01ce89db\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") "
Sep 10 17:30:48.877367 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877259    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbrdd\" (UniqueName: \"kubernetes.io/projected/a8df0756-9c18-41e3-a44e-10ea01ce89db-kube-api-access-cbrdd\") pod \"a8df0756-9c18-41e3-a44e-10ea01ce89db\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") "
Sep 10 17:30:48.877367 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877298    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-token\") pod \"a8df0756-9c18-41e3-a44e-10ea01ce89db\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") "
Sep 10 17:30:48.877367 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877300    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-data" (OuterVolumeSpecName: "istio-data") pod "a8df0756-9c18-41e3-a44e-10ea01ce89db" (UID: "a8df0756-9c18-41e3-a44e-10ea01ce89db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:30:48.877367 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877322    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-envoy\") pod \"a8df0756-9c18-41e3-a44e-10ea01ce89db\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") "
Sep 10 17:30:48.877573 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877380    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8df0756-9c18-41e3-a44e-10ea01ce89db-istiod-ca-cert\") pod \"a8df0756-9c18-41e3-a44e-10ea01ce89db\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") "
Sep 10 17:30:48.877573 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877405    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-credential-socket\") pod \"a8df0756-9c18-41e3-a44e-10ea01ce89db\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") "
Sep 10 17:30:48.877573 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877436    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a8df0756-9c18-41e3-a44e-10ea01ce89db" (UID: "a8df0756-9c18-41e3-a44e-10ea01ce89db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:30:48.877573 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877462    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-podinfo\") pod \"a8df0756-9c18-41e3-a44e-10ea01ce89db\" (UID: \"a8df0756-9c18-41e3-a44e-10ea01ce89db\") "
Sep 10 17:30:48.877573 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877545    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a8df0756-9c18-41e3-a44e-10ea01ce89db" (UID: "a8df0756-9c18-41e3-a44e-10ea01ce89db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:30:48.877813 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877641    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a8df0756-9c18-41e3-a44e-10ea01ce89db" (UID: "a8df0756-9c18-41e3-a44e-10ea01ce89db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:30:48.877813 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877718    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8df0756-9c18-41e3-a44e-10ea01ce89db-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a8df0756-9c18-41e3-a44e-10ea01ce89db" (UID: "a8df0756-9c18-41e3-a44e-10ea01ce89db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:30:48.877813 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877800    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8df0756-9c18-41e3-a44e-10ea01ce89db-istiod-ca-cert\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:48.877966 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877817    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-credential-socket\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:48.877966 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877831    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-data\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:48.877966 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877844    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-workload-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:48.877966 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.877855    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-workload-socket\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:48.879517 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.879480    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a8df0756-9c18-41e3-a44e-10ea01ce89db" (UID: "a8df0756-9c18-41e3-a44e-10ea01ce89db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:30:48.879602 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.879522    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8df0756-9c18-41e3-a44e-10ea01ce89db-kube-api-access-cbrdd" (OuterVolumeSpecName: "kube-api-access-cbrdd") pod "a8df0756-9c18-41e3-a44e-10ea01ce89db" (UID: "a8df0756-9c18-41e3-a44e-10ea01ce89db"). InnerVolumeSpecName "kube-api-access-cbrdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:30:48.879769 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.879748    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a8df0756-9c18-41e3-a44e-10ea01ce89db" (UID: "a8df0756-9c18-41e3-a44e-10ea01ce89db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 17:30:48.879769 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.879762    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-token" (OuterVolumeSpecName: "istio-token") pod "a8df0756-9c18-41e3-a44e-10ea01ce89db" (UID: "a8df0756-9c18-41e3-a44e-10ea01ce89db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:30:48.978496 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.978464    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbrdd\" (UniqueName: \"kubernetes.io/projected/a8df0756-9c18-41e3-a44e-10ea01ce89db-kube-api-access-cbrdd\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:48.978496 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.978491    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-token\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:48.978496 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.978500    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-envoy\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:48.978701 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:48.978509    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8df0756-9c18-41e3-a44e-10ea01ce89db-istio-podinfo\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:30:49.708669 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:49.708622    2566 generic.go:356] "Generic (PLEG): container finished" podID="a8df0756-9c18-41e3-a44e-10ea01ce89db" containerID="3800715e13c7fa1ce6afef8174805316f7a80b6462b80fe0bb7bce520feff6f4" exitCode=0
Sep 10 17:30:49.708854 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:49.708680    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"
Sep 10 17:30:49.708854 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:49.708711    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v" event={"ID":"a8df0756-9c18-41e3-a44e-10ea01ce89db","Type":"ContainerDied","Data":"3800715e13c7fa1ce6afef8174805316f7a80b6462b80fe0bb7bce520feff6f4"}
Sep 10 17:30:49.708854 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:49.708777    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v" event={"ID":"a8df0756-9c18-41e3-a44e-10ea01ce89db","Type":"ContainerDied","Data":"81dd135874c44394b56d625b5e3a54a84c57ebed9bab2a2dd605abc62181eda3"}
Sep 10 17:30:49.708854 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:49.708817    2566 scope.go:117] "RemoveContainer" containerID="3800715e13c7fa1ce6afef8174805316f7a80b6462b80fe0bb7bce520feff6f4"
Sep 10 17:30:49.719690 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:49.719531    2566 scope.go:117] "RemoveContainer" containerID="fbc62a6051fab4c14b13cb918e288f2c0363880d924b15f82e2b7e956d1c9e05"
Sep 10 17:30:49.727884 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:49.727868    2566 scope.go:117] "RemoveContainer" containerID="3800715e13c7fa1ce6afef8174805316f7a80b6462b80fe0bb7bce520feff6f4"
Sep 10 17:30:49.728183 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:30:49.728111    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3800715e13c7fa1ce6afef8174805316f7a80b6462b80fe0bb7bce520feff6f4\": container with ID starting with 3800715e13c7fa1ce6afef8174805316f7a80b6462b80fe0bb7bce520feff6f4 not found: ID does not exist" containerID="3800715e13c7fa1ce6afef8174805316f7a80b6462b80fe0bb7bce520feff6f4"
Sep 10 17:30:49.728228 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:49.728191    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3800715e13c7fa1ce6afef8174805316f7a80b6462b80fe0bb7bce520feff6f4"} err="failed to get container status \"3800715e13c7fa1ce6afef8174805316f7a80b6462b80fe0bb7bce520feff6f4\": rpc error: code = NotFound desc = could not find container \"3800715e13c7fa1ce6afef8174805316f7a80b6462b80fe0bb7bce520feff6f4\": container with ID starting with 3800715e13c7fa1ce6afef8174805316f7a80b6462b80fe0bb7bce520feff6f4 not found: ID does not exist"
Sep 10 17:30:49.728228 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:49.728210    2566 scope.go:117] "RemoveContainer" containerID="fbc62a6051fab4c14b13cb918e288f2c0363880d924b15f82e2b7e956d1c9e05"
Sep 10 17:30:49.728463 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:30:49.728442    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbc62a6051fab4c14b13cb918e288f2c0363880d924b15f82e2b7e956d1c9e05\": container with ID starting with fbc62a6051fab4c14b13cb918e288f2c0363880d924b15f82e2b7e956d1c9e05 not found: ID does not exist" containerID="fbc62a6051fab4c14b13cb918e288f2c0363880d924b15f82e2b7e956d1c9e05"
Sep 10 17:30:49.728517 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:49.728472    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbc62a6051fab4c14b13cb918e288f2c0363880d924b15f82e2b7e956d1c9e05"} err="failed to get container status \"fbc62a6051fab4c14b13cb918e288f2c0363880d924b15f82e2b7e956d1c9e05\": rpc error: code = NotFound desc = could not find container \"fbc62a6051fab4c14b13cb918e288f2c0363880d924b15f82e2b7e956d1c9e05\": container with ID starting with fbc62a6051fab4c14b13cb918e288f2c0363880d924b15f82e2b7e956d1c9e05 not found: ID does not exist"
Sep 10 17:30:49.742244 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:49.742217    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"]
Sep 10 17:30:49.745719 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:49.745699    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-dcc1cd6c-openshift-default-77d86d6c76-kpk4v"]
Sep 10 17:30:49.995139 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:49.995100    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:50.834951 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.834917    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8df0756-9c18-41e3-a44e-10ea01ce89db" path="/var/lib/kubelet/pods/a8df0756-9c18-41e3-a44e-10ea01ce89db/volumes"
Sep 10 17:30:50.958308 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958273    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"]
Sep 10 17:30:50.958697 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958683    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a70068e-8b2f-49f5-9893-80290b22b244" containerName="maas-api"
Sep 10 17:30:50.958697 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958698    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a70068e-8b2f-49f5-9893-80290b22b244" containerName="maas-api"
Sep 10 17:30:50.958808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958705    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff19c3ea-8ee9-455d-9989-a5a607dc6cc4" containerName="cleanup"
Sep 10 17:30:50.958808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958710    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff19c3ea-8ee9-455d-9989-a5a607dc6cc4" containerName="cleanup"
Sep 10 17:30:50.958808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958721    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="664fb2f5-14e8-4266-a702-e75751ff315c" containerName="cleanup"
Sep 10 17:30:50.958808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958727    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="664fb2f5-14e8-4266-a702-e75751ff315c" containerName="cleanup"
Sep 10 17:30:50.958808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958734    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e80813d-281d-4109-8e31-946428b49733" containerName="istio-proxy"
Sep 10 17:30:50.958808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958739    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e80813d-281d-4109-8e31-946428b49733" containerName="istio-proxy"
Sep 10 17:30:50.958808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958746    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8df0756-9c18-41e3-a44e-10ea01ce89db" containerName="istio-proxy"
Sep 10 17:30:50.958808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958751    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8df0756-9c18-41e3-a44e-10ea01ce89db" containerName="istio-proxy"
Sep 10 17:30:50.958808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958766    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8df0756-9c18-41e3-a44e-10ea01ce89db" containerName="istio-proxy"
Sep 10 17:30:50.958808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958771    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8df0756-9c18-41e3-a44e-10ea01ce89db" containerName="istio-proxy"
Sep 10 17:30:50.958808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958779    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a27d6f-48cf-40f5-925f-de78c5922e9b" containerName="maas-api"
Sep 10 17:30:50.958808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958784    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a27d6f-48cf-40f5-925f-de78c5922e9b" containerName="maas-api"
Sep 10 17:30:50.958808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958791    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97ba04cb-05e1-418a-8dac-e5b951f997ae" containerName="payload-pre-processing"
Sep 10 17:30:50.958808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958797    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="97ba04cb-05e1-418a-8dac-e5b951f997ae" containerName="payload-pre-processing"
Sep 10 17:30:50.959507 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958853    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8df0756-9c18-41e3-a44e-10ea01ce89db" containerName="istio-proxy"
Sep 10 17:30:50.959507 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958860    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a27d6f-48cf-40f5-925f-de78c5922e9b" containerName="maas-api"
Sep 10 17:30:50.959507 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958866    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a70068e-8b2f-49f5-9893-80290b22b244" containerName="maas-api"
Sep 10 17:30:50.959507 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958872    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff19c3ea-8ee9-455d-9989-a5a607dc6cc4" containerName="cleanup"
Sep 10 17:30:50.959507 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958877    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="664fb2f5-14e8-4266-a702-e75751ff315c" containerName="cleanup"
Sep 10 17:30:50.959507 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958882    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e80813d-281d-4109-8e31-946428b49733" containerName="istio-proxy"
Sep 10 17:30:50.959507 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958888    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="97ba04cb-05e1-418a-8dac-e5b951f997ae" containerName="payload-pre-processing"
Sep 10 17:30:50.959507 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.958895    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e80813d-281d-4109-8e31-946428b49733" containerName="istio-proxy"
Sep 10 17:30:50.963658 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.963634    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:30:50.966619 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.966594    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-9e9c14-serving-cert\""
Sep 10 17:30:50.966619 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.966613    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:30:50.966825 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.966681    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 17:30:50.967661 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.967641    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 17:30:50.967661 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.967653    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-2hfkp\""
Sep 10 17:30:50.979009 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.978987    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"]
Sep 10 17:30:50.996200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.996173    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbjz2\" (UniqueName: \"kubernetes.io/projected/f47ea149-6ee9-4147-8461-531de81f2e1b-kube-api-access-zbjz2\") pod \"maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd\" (UID: \"f47ea149-6ee9-4147-8461-531de81f2e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:30:50.996463 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.996219    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f47ea149-6ee9-4147-8461-531de81f2e1b-maas-api-tls\") pod \"maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd\" (UID: \"f47ea149-6ee9-4147-8461-531de81f2e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:30:50.996463 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.996326    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f47ea149-6ee9-4147-8461-531de81f2e1b-metrics-certs\") pod \"maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd\" (UID: \"f47ea149-6ee9-4147-8461-531de81f2e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:30:50.996463 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:50.996396    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f47ea149-6ee9-4147-8461-531de81f2e1b-openshift-service-ca\") pod \"maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd\" (UID: \"f47ea149-6ee9-4147-8461-531de81f2e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:30:51.028481 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:51.028447    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:51.097425 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:51.097350    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f47ea149-6ee9-4147-8461-531de81f2e1b-maas-api-tls\") pod \"maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd\" (UID: \"f47ea149-6ee9-4147-8461-531de81f2e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:30:51.097425 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:51.097405    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f47ea149-6ee9-4147-8461-531de81f2e1b-metrics-certs\") pod \"maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd\" (UID: \"f47ea149-6ee9-4147-8461-531de81f2e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:30:51.097588 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:51.097452    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f47ea149-6ee9-4147-8461-531de81f2e1b-openshift-service-ca\") pod \"maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd\" (UID: \"f47ea149-6ee9-4147-8461-531de81f2e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:30:51.097588 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:30:51.097501    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-serving-cert: secret "maas-api-e2e-explicit-9e9c14-serving-cert" not found
Sep 10 17:30:51.097588 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:51.097514    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbjz2\" (UniqueName: \"kubernetes.io/projected/f47ea149-6ee9-4147-8461-531de81f2e1b-kube-api-access-zbjz2\") pod \"maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd\" (UID: \"f47ea149-6ee9-4147-8461-531de81f2e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:30:51.097588 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:30:51.097553    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f47ea149-6ee9-4147-8461-531de81f2e1b-maas-api-tls podName:f47ea149-6ee9-4147-8461-531de81f2e1b nodeName:}" failed. No retries permitted until 2026-09-10 17:30:51.597537605 +0000 UTC m=+1055.278710526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f47ea149-6ee9-4147-8461-531de81f2e1b-maas-api-tls") pod "maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd" (UID: "f47ea149-6ee9-4147-8461-531de81f2e1b") : secret "maas-api-e2e-explicit-9e9c14-serving-cert" not found
Sep 10 17:30:51.098185 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:51.098167    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f47ea149-6ee9-4147-8461-531de81f2e1b-openshift-service-ca\") pod \"maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd\" (UID: \"f47ea149-6ee9-4147-8461-531de81f2e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:30:51.099853 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:51.099835    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f47ea149-6ee9-4147-8461-531de81f2e1b-metrics-certs\") pod \"maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd\" (UID: \"f47ea149-6ee9-4147-8461-531de81f2e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:30:51.111237 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:51.111204    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbjz2\" (UniqueName: \"kubernetes.io/projected/f47ea149-6ee9-4147-8461-531de81f2e1b-kube-api-access-zbjz2\") pod \"maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd\" (UID: \"f47ea149-6ee9-4147-8461-531de81f2e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:30:51.136793 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:51.136758    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:51.400648 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:51.400546    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:51.602609 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:51.602571    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f47ea149-6ee9-4147-8461-531de81f2e1b-maas-api-tls\") pod \"maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd\" (UID: \"f47ea149-6ee9-4147-8461-531de81f2e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:30:51.605081 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:51.605055    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f47ea149-6ee9-4147-8461-531de81f2e1b-maas-api-tls\") pod \"maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd\" (UID: \"f47ea149-6ee9-4147-8461-531de81f2e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:30:51.757774 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:51.757742    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:51.873878 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:51.873838    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:51.877235 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:51.877215    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:30:52.157778 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:52.157748    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"]
Sep 10 17:30:52.159210 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:30:52.159177    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf47ea149_6ee9_4147_8461_531de81f2e1b.slice/crio-1442cb067a77334b777af0c70573d9a06f75b1b0ea502ff28bc422a2cf85e0f4 WatchSource:0}: Error finding container 1442cb067a77334b777af0c70573d9a06f75b1b0ea502ff28bc422a2cf85e0f4: Status 404 returned error can't find the container with id 1442cb067a77334b777af0c70573d9a06f75b1b0ea502ff28bc422a2cf85e0f4
Sep 10 17:30:52.731241 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:52.731204    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd" event={"ID":"f47ea149-6ee9-4147-8461-531de81f2e1b","Type":"ContainerStarted","Data":"414d614c05eecad68fc31e49b02f8a7bfe98607d28caa28d5f2a0e42a1263250"}
Sep 10 17:30:52.731442 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:52.731263    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd" event={"ID":"f47ea149-6ee9-4147-8461-531de81f2e1b","Type":"ContainerStarted","Data":"1442cb067a77334b777af0c70573d9a06f75b1b0ea502ff28bc422a2cf85e0f4"}
Sep 10 17:30:52.731442 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:52.731324    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:30:52.762025 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:52.761971    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd" podStartSLOduration=2.362737101 podStartE2EDuration="2.761954213s" totalImagesPullingTime="399.217112ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:30:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:30:52.760646088 +0000 UTC m=+1056.441819013" watchObservedRunningTime="2026-09-10 17:30:52.761954213 +0000 UTC m=+1056.443127153"
Sep 10 17:30:54.119932 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:54.119888    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:54.224502 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:54.224467    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:54.332694 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:54.332652    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:54.461865 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:54.461779    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:54.855727 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:54.855696    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:30:58.743140 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:30:58.743097    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:31:13.904385 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:31:13.904294    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:31:14.825497 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:31:14.825458    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:31:15.732688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:31:15.732645    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:31:16.471196 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:31:16.471162    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:31:17.194385 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:31:17.194311    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:31:17.872392 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:31:17.872355    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:31:20.767184 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:31:20.767147    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:31:26.941974 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:31:26.941928    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:31:27.019417 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:31:27.019377    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:31:27.115327 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:31:27.115295    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:31:27.208833 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:31:27.208749    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:31:43.031455 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:31:43.031416    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:31:48.929786 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:31:48.929737    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:31:49.060630 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:31:49.060598    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:31:49.186090 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:31:49.185968    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:31:50.162217 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:31:50.162178    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:31:59.598790 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:31:59.598750    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:00.348305 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:00.348266    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:02.120337 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.120300    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"]
Sep 10 17:32:02.120947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.120926    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e80813d-281d-4109-8e31-946428b49733" containerName="istio-proxy"
Sep 10 17:32:02.121041 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.120950    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e80813d-281d-4109-8e31-946428b49733" containerName="istio-proxy"
Sep 10 17:32:02.121098 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.121081    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8df0756-9c18-41e3-a44e-10ea01ce89db" containerName="istio-proxy"
Sep 10 17:32:02.124326 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.124305    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.126892 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.126870    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-9e9c14\"/\"openshift-service-ca.crt\""
Sep 10 17:32:02.127677 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.127657    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-9e9c14\"/\"default-dockercfg-4xt5h\""
Sep 10 17:32:02.127901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.127881    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-9e9c14\"/\"kube-root-ca.crt\""
Sep 10 17:32:02.127965 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.127951    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-9e9c14\"/\"explicit-model-9e9c14-kserve-self-signed-certs\""
Sep 10 17:32:02.137598 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.137574    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"]
Sep 10 17:32:02.153112 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.153085    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:02.264255 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.264223    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-model-cache\") pod \"explicit-model-9e9c14-kserve-587cc6ffd7-fbv96\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") " pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.264456 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.264266    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/158647dc-8e4f-46c2-b543-ef968740de19-tls-certs\") pod \"explicit-model-9e9c14-kserve-587cc6ffd7-fbv96\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") " pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.264456 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.264308    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-home\") pod \"explicit-model-9e9c14-kserve-587cc6ffd7-fbv96\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") " pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.264456 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.264360    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-tmp-dir\") pod \"explicit-model-9e9c14-kserve-587cc6ffd7-fbv96\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") " pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.264456 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.264392    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-dshm\") pod \"explicit-model-9e9c14-kserve-587cc6ffd7-fbv96\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") " pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.264645 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.264456    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slcvk\" (UniqueName: \"kubernetes.io/projected/158647dc-8e4f-46c2-b543-ef968740de19-kube-api-access-slcvk\") pod \"explicit-model-9e9c14-kserve-587cc6ffd7-fbv96\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") " pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.365565 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.365527    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slcvk\" (UniqueName: \"kubernetes.io/projected/158647dc-8e4f-46c2-b543-ef968740de19-kube-api-access-slcvk\") pod \"explicit-model-9e9c14-kserve-587cc6ffd7-fbv96\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") " pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.365782 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.365595    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-model-cache\") pod \"explicit-model-9e9c14-kserve-587cc6ffd7-fbv96\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") " pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.365782 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.365618    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/158647dc-8e4f-46c2-b543-ef968740de19-tls-certs\") pod \"explicit-model-9e9c14-kserve-587cc6ffd7-fbv96\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") " pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.365782 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.365657    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-home\") pod \"explicit-model-9e9c14-kserve-587cc6ffd7-fbv96\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") " pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.365782 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.365696    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-tmp-dir\") pod \"explicit-model-9e9c14-kserve-587cc6ffd7-fbv96\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") " pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.365782 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.365734    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-dshm\") pod \"explicit-model-9e9c14-kserve-587cc6ffd7-fbv96\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") " pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.366153 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.366106    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-model-cache\") pod \"explicit-model-9e9c14-kserve-587cc6ffd7-fbv96\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") " pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.366222 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.366177    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-home\") pod \"explicit-model-9e9c14-kserve-587cc6ffd7-fbv96\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") " pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.366222 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.366185    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-tmp-dir\") pod \"explicit-model-9e9c14-kserve-587cc6ffd7-fbv96\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") " pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.368051 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.368033    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-dshm\") pod \"explicit-model-9e9c14-kserve-587cc6ffd7-fbv96\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") " pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.368353 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.368333    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/158647dc-8e4f-46c2-b543-ef968740de19-tls-certs\") pod \"explicit-model-9e9c14-kserve-587cc6ffd7-fbv96\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") " pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.374582 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.374524    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slcvk\" (UniqueName: \"kubernetes.io/projected/158647dc-8e4f-46c2-b543-ef968740de19-kube-api-access-slcvk\") pod \"explicit-model-9e9c14-kserve-587cc6ffd7-fbv96\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") " pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.433728 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.433689    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:02.709299 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:02.709267    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"]
Sep 10 17:32:02.709847 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:32:02.709817    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod158647dc_8e4f_46c2_b543_ef968740de19.slice/crio-62a8cf76267dc89240327a67bba58a2f136859cc046235b986a8972c6dda7cbe WatchSource:0}: Error finding container 62a8cf76267dc89240327a67bba58a2f136859cc046235b986a8972c6dda7cbe: Status 404 returned error can't find the container with id 62a8cf76267dc89240327a67bba58a2f136859cc046235b986a8972c6dda7cbe
Sep 10 17:32:03.172535 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:03.172498    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96" event={"ID":"158647dc-8e4f-46c2-b543-ef968740de19","Type":"ContainerStarted","Data":"1fde9918c71ca76b7046e953950313f79994fbf202e3a8fed31917a5a5c87406"}
Sep 10 17:32:03.172535 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:03.172538    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96" event={"ID":"158647dc-8e4f-46c2-b543-ef968740de19","Type":"ContainerStarted","Data":"62a8cf76267dc89240327a67bba58a2f136859cc046235b986a8972c6dda7cbe"}
Sep 10 17:32:03.193619 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:03.193570    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96" podStartSLOduration=1.193555938 podStartE2EDuration="1.193555938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:32:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:32:03.192053887 +0000 UTC m=+1126.873226866" watchObservedRunningTime="2026-09-10 17:32:03.193555938 +0000 UTC m=+1126.874728880"
Sep 10 17:32:06.243545 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:06.243505    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:06.349400 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:06.349368    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:06.458479 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:06.458433    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:06.555472 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:06.555385    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:07.184866 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:07.184826    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:07.397392 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:07.397355    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:07.549517 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:07.549481    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:08.890790 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:08.890729    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:11.815667 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:11.815636    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"]
Sep 10 17:32:11.837911 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:11.837878    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:11.938102 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:11.938065    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj"]
Sep 10 17:32:11.941833 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:11.941817    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj"
Sep 10 17:32:11.944999 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:11.944984    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxddn\""
Sep 10 17:32:11.953740 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:11.953715    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj"]
Sep 10 17:32:11.956102 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:11.956076    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/841c1937-25b7-48fe-b654-f9b8d09bc3fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj\" (UID: \"841c1937-25b7-48fe-b654-f9b8d09bc3fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj"
Sep 10 17:32:12.056767 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:12.056730    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/841c1937-25b7-48fe-b654-f9b8d09bc3fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj\" (UID: \"841c1937-25b7-48fe-b654-f9b8d09bc3fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj"
Sep 10 17:32:12.057383 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:12.057362    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/841c1937-25b7-48fe-b654-f9b8d09bc3fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj\" (UID: \"841c1937-25b7-48fe-b654-f9b8d09bc3fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj"
Sep 10 17:32:12.253795 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:12.253761    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj"
Sep 10 17:32:12.434225 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:12.434189    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:12.533163 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:12.533119    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj"]
Sep 10 17:32:12.534327 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:32:12.534304    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod841c1937_25b7_48fe_b654_f9b8d09bc3fd.slice/crio-9005db6b4a4d23205875a9b7ef15d171ccd55bd82eba6aeb63f8716fe2e39e8e WatchSource:0}: Error finding container 9005db6b4a4d23205875a9b7ef15d171ccd55bd82eba6aeb63f8716fe2e39e8e: Status 404 returned error can't find the container with id 9005db6b4a4d23205875a9b7ef15d171ccd55bd82eba6aeb63f8716fe2e39e8e
Sep 10 17:32:12.981695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:12.981663    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:13.234509 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:13.234415    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj" event={"ID":"841c1937-25b7-48fe-b654-f9b8d09bc3fd","Type":"ContainerStarted","Data":"eef3a15e2c32e701e322c60addfc21f1018928caedb326df9f4f541db5632189"}
Sep 10 17:32:13.234509 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:13.234478    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj" event={"ID":"841c1937-25b7-48fe-b654-f9b8d09bc3fd","Type":"ContainerStarted","Data":"9005db6b4a4d23205875a9b7ef15d171ccd55bd82eba6aeb63f8716fe2e39e8e"}
Sep 10 17:32:13.257432 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:13.257375    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj" podStartSLOduration=2.257360105 podStartE2EDuration="2.257360105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:32:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:32:13.254991227 +0000 UTC m=+1136.936164171" watchObservedRunningTime="2026-09-10 17:32:13.257360105 +0000 UTC m=+1136.938533063"
Sep 10 17:32:13.827977 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:13.827939    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:14.242119 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:14.242084    2566 generic.go:356] "Generic (PLEG): container finished" podID="841c1937-25b7-48fe-b654-f9b8d09bc3fd" containerID="eef3a15e2c32e701e322c60addfc21f1018928caedb326df9f4f541db5632189" exitCode=0
Sep 10 17:32:14.242598 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:14.242166    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj" event={"ID":"841c1937-25b7-48fe-b654-f9b8d09bc3fd","Type":"ContainerDied","Data":"eef3a15e2c32e701e322c60addfc21f1018928caedb326df9f4f541db5632189"}
Sep 10 17:32:14.745083 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:14.745051    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:15.382901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:15.382877    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj"
Sep 10 17:32:15.491153 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:15.491089    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/841c1937-25b7-48fe-b654-f9b8d09bc3fd-maas-api-service-ca\") pod \"841c1937-25b7-48fe-b654-f9b8d09bc3fd\" (UID: \"841c1937-25b7-48fe-b654-f9b8d09bc3fd\") "
Sep 10 17:32:15.491494 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:15.491471    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/841c1937-25b7-48fe-b654-f9b8d09bc3fd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "841c1937-25b7-48fe-b654-f9b8d09bc3fd" (UID: "841c1937-25b7-48fe-b654-f9b8d09bc3fd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:32:15.592663 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:15.592581    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/841c1937-25b7-48fe-b654-f9b8d09bc3fd-maas-api-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:32:15.982628 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:15.982538    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:16.255778 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:16.255739    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj" event={"ID":"841c1937-25b7-48fe-b654-f9b8d09bc3fd","Type":"ContainerDied","Data":"9005db6b4a4d23205875a9b7ef15d171ccd55bd82eba6aeb63f8716fe2e39e8e"}
Sep 10 17:32:16.255946 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:16.255788    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9005db6b4a4d23205875a9b7ef15d171ccd55bd82eba6aeb63f8716fe2e39e8e"
Sep 10 17:32:16.255946 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:16.255753    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj"
Sep 10 17:32:16.895823 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:16.895778    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:17.006665 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:17.006630    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:17.630218 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:17.630173    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:22.607495 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:22.607462    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"]
Sep 10 17:32:22.607933 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:22.607702    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd" podUID="f47ea149-6ee9-4147-8461-531de81f2e1b" containerName="maas-api" containerID="cri-o://414d614c05eecad68fc31e49b02f8a7bfe98607d28caa28d5f2a0e42a1263250" gracePeriod=30
Sep 10 17:32:22.649461 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:22.649430    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:22.744964 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:22.744920    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:22.845389 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:22.845361    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:22.862033 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:22.861972    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:32:22.911919 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:22.911667    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj"]
Sep 10 17:32:22.916111 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:22.916081    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9e9c14-thrnj"]
Sep 10 17:32:22.918215 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:22.918177    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:22.969294 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:22.969260    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f47ea149-6ee9-4147-8461-531de81f2e1b-metrics-certs\") pod \"f47ea149-6ee9-4147-8461-531de81f2e1b\" (UID: \"f47ea149-6ee9-4147-8461-531de81f2e1b\") "
Sep 10 17:32:22.969452 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:22.969371    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbjz2\" (UniqueName: \"kubernetes.io/projected/f47ea149-6ee9-4147-8461-531de81f2e1b-kube-api-access-zbjz2\") pod \"f47ea149-6ee9-4147-8461-531de81f2e1b\" (UID: \"f47ea149-6ee9-4147-8461-531de81f2e1b\") "
Sep 10 17:32:22.969452 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:22.969428    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f47ea149-6ee9-4147-8461-531de81f2e1b-maas-api-tls\") pod \"f47ea149-6ee9-4147-8461-531de81f2e1b\" (UID: \"f47ea149-6ee9-4147-8461-531de81f2e1b\") "
Sep 10 17:32:22.969588 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:22.969519    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f47ea149-6ee9-4147-8461-531de81f2e1b-openshift-service-ca\") pod \"f47ea149-6ee9-4147-8461-531de81f2e1b\" (UID: \"f47ea149-6ee9-4147-8461-531de81f2e1b\") "
Sep 10 17:32:22.969989 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:22.969950    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f47ea149-6ee9-4147-8461-531de81f2e1b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f47ea149-6ee9-4147-8461-531de81f2e1b" (UID: "f47ea149-6ee9-4147-8461-531de81f2e1b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:32:22.971789 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:22.971755    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f47ea149-6ee9-4147-8461-531de81f2e1b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f47ea149-6ee9-4147-8461-531de81f2e1b" (UID: "f47ea149-6ee9-4147-8461-531de81f2e1b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:32:22.972098 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:22.972073    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f47ea149-6ee9-4147-8461-531de81f2e1b-kube-api-access-zbjz2" (OuterVolumeSpecName: "kube-api-access-zbjz2") pod "f47ea149-6ee9-4147-8461-531de81f2e1b" (UID: "f47ea149-6ee9-4147-8461-531de81f2e1b"). InnerVolumeSpecName "kube-api-access-zbjz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:32:22.972244 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:22.972214    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f47ea149-6ee9-4147-8461-531de81f2e1b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f47ea149-6ee9-4147-8461-531de81f2e1b" (UID: "f47ea149-6ee9-4147-8461-531de81f2e1b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:32:23.070443 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:23.070409    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbjz2\" (UniqueName: \"kubernetes.io/projected/f47ea149-6ee9-4147-8461-531de81f2e1b-kube-api-access-zbjz2\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:32:23.070443 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:23.070438    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f47ea149-6ee9-4147-8461-531de81f2e1b-maas-api-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:32:23.070443 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:23.070449    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f47ea149-6ee9-4147-8461-531de81f2e1b-openshift-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:32:23.070688 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:23.070458    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f47ea149-6ee9-4147-8461-531de81f2e1b-metrics-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:32:23.272477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:23.272445    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:23.299303 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:23.299270    2566 generic.go:356] "Generic (PLEG): container finished" podID="f47ea149-6ee9-4147-8461-531de81f2e1b" containerID="414d614c05eecad68fc31e49b02f8a7bfe98607d28caa28d5f2a0e42a1263250" exitCode=0
Sep 10 17:32:23.299463 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:23.299315    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"
Sep 10 17:32:23.299463 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:23.299356    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd" event={"ID":"f47ea149-6ee9-4147-8461-531de81f2e1b","Type":"ContainerDied","Data":"414d614c05eecad68fc31e49b02f8a7bfe98607d28caa28d5f2a0e42a1263250"}
Sep 10 17:32:23.299463 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:23.299414    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd" event={"ID":"f47ea149-6ee9-4147-8461-531de81f2e1b","Type":"ContainerDied","Data":"1442cb067a77334b777af0c70573d9a06f75b1b0ea502ff28bc422a2cf85e0f4"}
Sep 10 17:32:23.299463 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:23.299443    2566 scope.go:117] "RemoveContainer" containerID="414d614c05eecad68fc31e49b02f8a7bfe98607d28caa28d5f2a0e42a1263250"
Sep 10 17:32:23.312593 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:23.312572    2566 scope.go:117] "RemoveContainer" containerID="414d614c05eecad68fc31e49b02f8a7bfe98607d28caa28d5f2a0e42a1263250"
Sep 10 17:32:23.312932 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:32:23.312908    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"414d614c05eecad68fc31e49b02f8a7bfe98607d28caa28d5f2a0e42a1263250\": container with ID starting with 414d614c05eecad68fc31e49b02f8a7bfe98607d28caa28d5f2a0e42a1263250 not found: ID does not exist" containerID="414d614c05eecad68fc31e49b02f8a7bfe98607d28caa28d5f2a0e42a1263250"
Sep 10 17:32:23.313044 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:23.312943    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"414d614c05eecad68fc31e49b02f8a7bfe98607d28caa28d5f2a0e42a1263250"} err="failed to get container status \"414d614c05eecad68fc31e49b02f8a7bfe98607d28caa28d5f2a0e42a1263250\": rpc error: code = NotFound desc = could not find container \"414d614c05eecad68fc31e49b02f8a7bfe98607d28caa28d5f2a0e42a1263250\": container with ID starting with 414d614c05eecad68fc31e49b02f8a7bfe98607d28caa28d5f2a0e42a1263250 not found: ID does not exist"
Sep 10 17:32:23.326383 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:23.326355    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"]
Sep 10 17:32:23.334987 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:23.334958    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-9e9c14-576bfdf887-t2swd"]
Sep 10 17:32:23.351520 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:23.351489    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:24.252392 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:24.252355    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:24.713721 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:24.713628    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:24.818882 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:24.818846    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:24.834278 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:24.834249    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="841c1937-25b7-48fe-b654-f9b8d09bc3fd" path="/var/lib/kubelet/pods/841c1937-25b7-48fe-b654-f9b8d09bc3fd/volumes"
Sep 10 17:32:24.834667 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:24.834654    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f47ea149-6ee9-4147-8461-531de81f2e1b" path="/var/lib/kubelet/pods/f47ea149-6ee9-4147-8461-531de81f2e1b/volumes"
Sep 10 17:32:24.940562 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:24.940519    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:25.182812 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:25.182772    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:25.296283 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:25.296247    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:25.648957 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:25.648916    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:26.836399 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:26.836353    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96" podUID="158647dc-8e4f-46c2-b543-ef968740de19" containerName="main" containerID="cri-o://1fde9918c71ca76b7046e953950313f79994fbf202e3a8fed31917a5a5c87406" gracePeriod=45
Sep 10 17:32:27.089446 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.089378    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:27.212863 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.212828    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-home\") pod \"158647dc-8e4f-46c2-b543-ef968740de19\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") "
Sep 10 17:32:27.213047 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.212891    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-tmp-dir\") pod \"158647dc-8e4f-46c2-b543-ef968740de19\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") "
Sep 10 17:32:27.213047 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.212999    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slcvk\" (UniqueName: \"kubernetes.io/projected/158647dc-8e4f-46c2-b543-ef968740de19-kube-api-access-slcvk\") pod \"158647dc-8e4f-46c2-b543-ef968740de19\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") "
Sep 10 17:32:27.213266 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.213059    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/158647dc-8e4f-46c2-b543-ef968740de19-tls-certs\") pod \"158647dc-8e4f-46c2-b543-ef968740de19\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") "
Sep 10 17:32:27.213266 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.213185    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-model-cache\") pod \"158647dc-8e4f-46c2-b543-ef968740de19\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") "
Sep 10 17:32:27.213266 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.213120    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-home" (OuterVolumeSpecName: "home") pod "158647dc-8e4f-46c2-b543-ef968740de19" (UID: "158647dc-8e4f-46c2-b543-ef968740de19"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:32:27.213266 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.213207    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "158647dc-8e4f-46c2-b543-ef968740de19" (UID: "158647dc-8e4f-46c2-b543-ef968740de19"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:32:27.213483 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.213289    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-dshm\") pod \"158647dc-8e4f-46c2-b543-ef968740de19\" (UID: \"158647dc-8e4f-46c2-b543-ef968740de19\") "
Sep 10 17:32:27.213541 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.213487    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-model-cache" (OuterVolumeSpecName: "model-cache") pod "158647dc-8e4f-46c2-b543-ef968740de19" (UID: "158647dc-8e4f-46c2-b543-ef968740de19"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:32:27.213715 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.213690    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-home\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:32:27.213715 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.213717    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-tmp-dir\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:32:27.213922 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.213733    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-model-cache\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:32:27.215459 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.215425    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/158647dc-8e4f-46c2-b543-ef968740de19-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "158647dc-8e4f-46c2-b543-ef968740de19" (UID: "158647dc-8e4f-46c2-b543-ef968740de19"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:32:27.215459 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.215440    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-dshm" (OuterVolumeSpecName: "dshm") pod "158647dc-8e4f-46c2-b543-ef968740de19" (UID: "158647dc-8e4f-46c2-b543-ef968740de19"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:32:27.215636 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.215538    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/158647dc-8e4f-46c2-b543-ef968740de19-kube-api-access-slcvk" (OuterVolumeSpecName: "kube-api-access-slcvk") pod "158647dc-8e4f-46c2-b543-ef968740de19" (UID: "158647dc-8e4f-46c2-b543-ef968740de19"). InnerVolumeSpecName "kube-api-access-slcvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:32:27.314493 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.314457    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/158647dc-8e4f-46c2-b543-ef968740de19-dshm\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:32:27.314493 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.314488    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slcvk\" (UniqueName: \"kubernetes.io/projected/158647dc-8e4f-46c2-b543-ef968740de19-kube-api-access-slcvk\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:32:27.314493 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.314499    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/158647dc-8e4f-46c2-b543-ef968740de19-tls-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:32:27.327331 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.327295    2566 generic.go:356] "Generic (PLEG): container finished" podID="158647dc-8e4f-46c2-b543-ef968740de19" containerID="1fde9918c71ca76b7046e953950313f79994fbf202e3a8fed31917a5a5c87406" exitCode=0
Sep 10 17:32:27.327485 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.327355    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"
Sep 10 17:32:27.327485 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.327393    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96" event={"ID":"158647dc-8e4f-46c2-b543-ef968740de19","Type":"ContainerDied","Data":"1fde9918c71ca76b7046e953950313f79994fbf202e3a8fed31917a5a5c87406"}
Sep 10 17:32:27.327485 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.327449    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96" event={"ID":"158647dc-8e4f-46c2-b543-ef968740de19","Type":"ContainerDied","Data":"62a8cf76267dc89240327a67bba58a2f136859cc046235b986a8972c6dda7cbe"}
Sep 10 17:32:27.327485 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.327471    2566 scope.go:117] "RemoveContainer" containerID="1fde9918c71ca76b7046e953950313f79994fbf202e3a8fed31917a5a5c87406"
Sep 10 17:32:27.337753 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.337731    2566 scope.go:117] "RemoveContainer" containerID="1fde9918c71ca76b7046e953950313f79994fbf202e3a8fed31917a5a5c87406"
Sep 10 17:32:27.338043 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:32:27.338024    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fde9918c71ca76b7046e953950313f79994fbf202e3a8fed31917a5a5c87406\": container with ID starting with 1fde9918c71ca76b7046e953950313f79994fbf202e3a8fed31917a5a5c87406 not found: ID does not exist" containerID="1fde9918c71ca76b7046e953950313f79994fbf202e3a8fed31917a5a5c87406"
Sep 10 17:32:27.338118 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.338049    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fde9918c71ca76b7046e953950313f79994fbf202e3a8fed31917a5a5c87406"} err="failed to get container status \"1fde9918c71ca76b7046e953950313f79994fbf202e3a8fed31917a5a5c87406\": rpc error: code = NotFound desc = could not find container \"1fde9918c71ca76b7046e953950313f79994fbf202e3a8fed31917a5a5c87406\": container with ID starting with 1fde9918c71ca76b7046e953950313f79994fbf202e3a8fed31917a5a5c87406 not found: ID does not exist"
Sep 10 17:32:27.350269 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.350207    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"]
Sep 10 17:32:27.353928 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.353905    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-9e9c14/explicit-model-9e9c14-kserve-587cc6ffd7-fbv96"]
Sep 10 17:32:27.841488 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:27.841454    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:28.497476 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:28.497431    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:28.612917 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:28.612876    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:28.751879 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:28.751791    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:28.834830 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:28.834788    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="158647dc-8e4f-46c2-b543-ef968740de19" path="/var/lib/kubelet/pods/158647dc-8e4f-46c2-b543-ef968740de19/volumes"
Sep 10 17:32:29.702551 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:29.702519    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:30.500614 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.500569    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:30.587310 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.587266    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:30.771917 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.771822    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"]
Sep 10 17:32:30.772815 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.772789    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="158647dc-8e4f-46c2-b543-ef968740de19" containerName="main"
Sep 10 17:32:30.772815 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.772814    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="158647dc-8e4f-46c2-b543-ef968740de19" containerName="main"
Sep 10 17:32:30.772983 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.772880    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f47ea149-6ee9-4147-8461-531de81f2e1b" containerName="maas-api"
Sep 10 17:32:30.772983 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.772890    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f47ea149-6ee9-4147-8461-531de81f2e1b" containerName="maas-api"
Sep 10 17:32:30.772983 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.772923    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="841c1937-25b7-48fe-b654-f9b8d09bc3fd" containerName="revoke-keys"
Sep 10 17:32:30.772983 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.772931    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="841c1937-25b7-48fe-b654-f9b8d09bc3fd" containerName="revoke-keys"
Sep 10 17:32:30.773207 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.773104    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="841c1937-25b7-48fe-b654-f9b8d09bc3fd" containerName="revoke-keys"
Sep 10 17:32:30.773207 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.773140    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f47ea149-6ee9-4147-8461-531de81f2e1b" containerName="maas-api"
Sep 10 17:32:30.773207 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.773149    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="158647dc-8e4f-46c2-b543-ef968740de19" containerName="main"
Sep 10 17:32:30.779021 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.778997    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"
Sep 10 17:32:30.786087 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.786064    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:32:30.787500 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.787474    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-2hfkp\""
Sep 10 17:32:30.787614 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.787583    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 17:32:30.787614 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.787593    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 17:32:30.787816 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.787797    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-851d1193-serving-cert\""
Sep 10 17:32:30.799355 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.799333    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"]
Sep 10 17:32:30.848283 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.848245    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7tgz\" (UniqueName: \"kubernetes.io/projected/b813802b-398c-4cf0-84e1-17c63c1ea9d7-kube-api-access-h7tgz\") pod \"maas-api-e2e-mt-851d1193-6bbdf665f-69fth\" (UID: \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"
Sep 10 17:32:30.848432 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.848353    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b813802b-398c-4cf0-84e1-17c63c1ea9d7-metrics-certs\") pod \"maas-api-e2e-mt-851d1193-6bbdf665f-69fth\" (UID: \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"
Sep 10 17:32:30.848432 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.848411    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b813802b-398c-4cf0-84e1-17c63c1ea9d7-openshift-service-ca\") pod \"maas-api-e2e-mt-851d1193-6bbdf665f-69fth\" (UID: \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"
Sep 10 17:32:30.848532 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.848503    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b813802b-398c-4cf0-84e1-17c63c1ea9d7-maas-api-tls\") pod \"maas-api-e2e-mt-851d1193-6bbdf665f-69fth\" (UID: \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"
Sep 10 17:32:30.858139 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.858096    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:30.940863 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.940822    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:30.949668 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.949640    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7tgz\" (UniqueName: \"kubernetes.io/projected/b813802b-398c-4cf0-84e1-17c63c1ea9d7-kube-api-access-h7tgz\") pod \"maas-api-e2e-mt-851d1193-6bbdf665f-69fth\" (UID: \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"
Sep 10 17:32:30.949808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.949697    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b813802b-398c-4cf0-84e1-17c63c1ea9d7-metrics-certs\") pod \"maas-api-e2e-mt-851d1193-6bbdf665f-69fth\" (UID: \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"
Sep 10 17:32:30.949808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.949731    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b813802b-398c-4cf0-84e1-17c63c1ea9d7-openshift-service-ca\") pod \"maas-api-e2e-mt-851d1193-6bbdf665f-69fth\" (UID: \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"
Sep 10 17:32:30.949808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.949789    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b813802b-398c-4cf0-84e1-17c63c1ea9d7-maas-api-tls\") pod \"maas-api-e2e-mt-851d1193-6bbdf665f-69fth\" (UID: \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"
Sep 10 17:32:30.950360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.950338    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b813802b-398c-4cf0-84e1-17c63c1ea9d7-openshift-service-ca\") pod \"maas-api-e2e-mt-851d1193-6bbdf665f-69fth\" (UID: \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"
Sep 10 17:32:30.952190 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.952171    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b813802b-398c-4cf0-84e1-17c63c1ea9d7-maas-api-tls\") pod \"maas-api-e2e-mt-851d1193-6bbdf665f-69fth\" (UID: \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"
Sep 10 17:32:30.952345 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.952327    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b813802b-398c-4cf0-84e1-17c63c1ea9d7-metrics-certs\") pod \"maas-api-e2e-mt-851d1193-6bbdf665f-69fth\" (UID: \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"
Sep 10 17:32:30.957890 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:30.957869    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7tgz\" (UniqueName: \"kubernetes.io/projected/b813802b-398c-4cf0-84e1-17c63c1ea9d7-kube-api-access-h7tgz\") pod \"maas-api-e2e-mt-851d1193-6bbdf665f-69fth\" (UID: \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"
Sep 10 17:32:31.091366 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:31.091273    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"
Sep 10 17:32:31.270029 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:31.269990    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:31.370993 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:31.370968    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"]
Sep 10 17:32:31.372203 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:32:31.372180    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb813802b_398c_4cf0_84e1_17c63c1ea9d7.slice/crio-c814462c2ddba4dbf84a33cc2bfbdc6ad504b4509efb6a781c67d530d540d7e0 WatchSource:0}: Error finding container c814462c2ddba4dbf84a33cc2bfbdc6ad504b4509efb6a781c67d530d540d7e0: Status 404 returned error can't find the container with id c814462c2ddba4dbf84a33cc2bfbdc6ad504b4509efb6a781c67d530d540d7e0
Sep 10 17:32:31.373586 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:31.373571    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 17:32:31.549818 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:31.549779    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:31.644596 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:31.644507    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:32.065440 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:32.065391    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:32.359821 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:32.359729    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth" event={"ID":"b813802b-398c-4cf0-84e1-17c63c1ea9d7","Type":"ContainerStarted","Data":"a6380989311c0860b779fcc6d61b34436c95f106b8cee0df65241f1311356ffa"}
Sep 10 17:32:32.359821 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:32.359796    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"
Sep 10 17:32:32.359821 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:32.359819    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth" event={"ID":"b813802b-398c-4cf0-84e1-17c63c1ea9d7","Type":"ContainerStarted","Data":"c814462c2ddba4dbf84a33cc2bfbdc6ad504b4509efb6a781c67d530d540d7e0"}
Sep 10 17:32:32.383831 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:32.383781    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth" podStartSLOduration=1.946340782 podStartE2EDuration="2.383764574s" totalImagesPullingTime="437.423792ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:32:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:32:32.381895834 +0000 UTC m=+1156.063068776" watchObservedRunningTime="2026-09-10 17:32:32.383764574 +0000 UTC m=+1156.064937569"
Sep 10 17:32:32.952326 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:32.952282    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:33.059972 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:33.059938    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:33.162183 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:33.162151    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:35.174058 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:35.174021    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:38.371733 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:38.371704    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"
Sep 10 17:32:40.003836 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:40.003742    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:53.430502 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:53.430461    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:54.056738 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:54.056703    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:54.172080 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:54.172036    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:54.537536 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:54.537501    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:55.118315 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:55.118281    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:55.358290 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:55.358258    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:55.805830 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:55.805790    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:56.238478 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:56.238385    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:32:57.159610 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:32:57.159551    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:00.602493 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:00.602441    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:00.812559 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:00.812518    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:00.937189 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:00.937085    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-851d1193-kqnnh"]
Sep 10 17:33:00.941265 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:00.941246    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-851d1193-kqnnh"
Sep 10 17:33:00.943736 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:00.943714    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxddn\""
Sep 10 17:33:00.950251 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:00.950215    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-851d1193-kqnnh"]
Sep 10 17:33:01.042606 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:01.042565    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5f39e1-6f18-48c8-98ef-cd276ab331f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-851d1193-kqnnh\" (UID: \"3f5f39e1-6f18-48c8-98ef-cd276ab331f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-851d1193-kqnnh"
Sep 10 17:33:01.143687 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:01.143648    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5f39e1-6f18-48c8-98ef-cd276ab331f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-851d1193-kqnnh\" (UID: \"3f5f39e1-6f18-48c8-98ef-cd276ab331f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-851d1193-kqnnh"
Sep 10 17:33:01.144319 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:01.144300    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5f39e1-6f18-48c8-98ef-cd276ab331f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-851d1193-kqnnh\" (UID: \"3f5f39e1-6f18-48c8-98ef-cd276ab331f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-851d1193-kqnnh"
Sep 10 17:33:01.254256 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:01.254225    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-851d1193-kqnnh"
Sep 10 17:33:01.528293 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:01.528271    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-851d1193-kqnnh"]
Sep 10 17:33:01.530088 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:33:01.530063    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f5f39e1_6f18_48c8_98ef_cd276ab331f8.slice/crio-ab4023146600ad6c5ed9f3aac6972a54ad4f9af8fd24dd47ed210f01cd0c4ecf WatchSource:0}: Error finding container ab4023146600ad6c5ed9f3aac6972a54ad4f9af8fd24dd47ed210f01cd0c4ecf: Status 404 returned error can't find the container with id ab4023146600ad6c5ed9f3aac6972a54ad4f9af8fd24dd47ed210f01cd0c4ecf
Sep 10 17:33:01.539977 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:01.539950    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-851d1193-kqnnh" event={"ID":"3f5f39e1-6f18-48c8-98ef-cd276ab331f8","Type":"ContainerStarted","Data":"ab4023146600ad6c5ed9f3aac6972a54ad4f9af8fd24dd47ed210f01cd0c4ecf"}
Sep 10 17:33:01.748664 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:01.748621    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:02.548714 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:02.548677    2566 generic.go:356] "Generic (PLEG): container finished" podID="3f5f39e1-6f18-48c8-98ef-cd276ab331f8" containerID="c578bc70ca48f7cc9892d760635754b215d3edb629fd9962d99b07663860fec3" exitCode=0
Sep 10 17:33:02.548890 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:02.548762    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-851d1193-kqnnh" event={"ID":"3f5f39e1-6f18-48c8-98ef-cd276ab331f8","Type":"ContainerDied","Data":"c578bc70ca48f7cc9892d760635754b215d3edb629fd9962d99b07663860fec3"}
Sep 10 17:33:02.665367 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:02.665332    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:03.409400 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:03.409363    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:03.685162 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:03.685115    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-851d1193-kqnnh"
Sep 10 17:33:03.768417 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:03.768384    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5f39e1-6f18-48c8-98ef-cd276ab331f8-maas-api-service-ca\") pod \"3f5f39e1-6f18-48c8-98ef-cd276ab331f8\" (UID: \"3f5f39e1-6f18-48c8-98ef-cd276ab331f8\") "
Sep 10 17:33:03.768788 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:03.768756    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f5f39e1-6f18-48c8-98ef-cd276ab331f8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3f5f39e1-6f18-48c8-98ef-cd276ab331f8" (UID: "3f5f39e1-6f18-48c8-98ef-cd276ab331f8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:33:03.869505 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:03.869471    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5f39e1-6f18-48c8-98ef-cd276ab331f8-maas-api-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:33:04.344728 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:04.344693    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:04.563041 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:04.563014    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-851d1193-kqnnh"
Sep 10 17:33:04.563425 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:04.563043    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-851d1193-kqnnh" event={"ID":"3f5f39e1-6f18-48c8-98ef-cd276ab331f8","Type":"ContainerDied","Data":"ab4023146600ad6c5ed9f3aac6972a54ad4f9af8fd24dd47ed210f01cd0c4ecf"}
Sep 10 17:33:04.563425 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:04.563084    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab4023146600ad6c5ed9f3aac6972a54ad4f9af8fd24dd47ed210f01cd0c4ecf"
Sep 10 17:33:11.435894 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.435852    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"]
Sep 10 17:33:11.436395 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.436111    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth" podUID="b813802b-398c-4cf0-84e1-17c63c1ea9d7" containerName="maas-api" containerID="cri-o://a6380989311c0860b779fcc6d61b34436c95f106b8cee0df65241f1311356ffa" gracePeriod=30
Sep 10 17:33:11.463388 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.463360    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:11.563950 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.563908    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:11.611426 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.611271    2566 generic.go:356] "Generic (PLEG): container finished" podID="b813802b-398c-4cf0-84e1-17c63c1ea9d7" containerID="a6380989311c0860b779fcc6d61b34436c95f106b8cee0df65241f1311356ffa" exitCode=0
Sep 10 17:33:11.611426 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.611364    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth" event={"ID":"b813802b-398c-4cf0-84e1-17c63c1ea9d7","Type":"ContainerDied","Data":"a6380989311c0860b779fcc6d61b34436c95f106b8cee0df65241f1311356ffa"}
Sep 10 17:33:11.663842 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.663802    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:11.671605 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.671582    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-851d1193-kqnnh"]
Sep 10 17:33:11.675364 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.675344    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-851d1193-kqnnh"]
Sep 10 17:33:11.687231 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.687178    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"
Sep 10 17:33:11.769001 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.768971    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:11.849881 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.849850    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b813802b-398c-4cf0-84e1-17c63c1ea9d7-metrics-certs\") pod \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\" (UID: \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\") "
Sep 10 17:33:11.850044 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.850031    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b813802b-398c-4cf0-84e1-17c63c1ea9d7-maas-api-tls\") pod \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\" (UID: \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\") "
Sep 10 17:33:11.850103 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.850080    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b813802b-398c-4cf0-84e1-17c63c1ea9d7-openshift-service-ca\") pod \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\" (UID: \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\") "
Sep 10 17:33:11.850196 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.850105    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7tgz\" (UniqueName: \"kubernetes.io/projected/b813802b-398c-4cf0-84e1-17c63c1ea9d7-kube-api-access-h7tgz\") pod \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\" (UID: \"b813802b-398c-4cf0-84e1-17c63c1ea9d7\") "
Sep 10 17:33:11.850438 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.850414    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b813802b-398c-4cf0-84e1-17c63c1ea9d7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b813802b-398c-4cf0-84e1-17c63c1ea9d7" (UID: "b813802b-398c-4cf0-84e1-17c63c1ea9d7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:33:11.851964 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.851938    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b813802b-398c-4cf0-84e1-17c63c1ea9d7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b813802b-398c-4cf0-84e1-17c63c1ea9d7" (UID: "b813802b-398c-4cf0-84e1-17c63c1ea9d7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:33:11.852096 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.852079    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b813802b-398c-4cf0-84e1-17c63c1ea9d7-kube-api-access-h7tgz" (OuterVolumeSpecName: "kube-api-access-h7tgz") pod "b813802b-398c-4cf0-84e1-17c63c1ea9d7" (UID: "b813802b-398c-4cf0-84e1-17c63c1ea9d7"). InnerVolumeSpecName "kube-api-access-h7tgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:33:11.852267 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.852243    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b813802b-398c-4cf0-84e1-17c63c1ea9d7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b813802b-398c-4cf0-84e1-17c63c1ea9d7" (UID: "b813802b-398c-4cf0-84e1-17c63c1ea9d7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:33:11.951533 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.951451    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b813802b-398c-4cf0-84e1-17c63c1ea9d7-maas-api-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:33:11.951533 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.951478    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b813802b-398c-4cf0-84e1-17c63c1ea9d7-openshift-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:33:11.951533 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.951489    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7tgz\" (UniqueName: \"kubernetes.io/projected/b813802b-398c-4cf0-84e1-17c63c1ea9d7-kube-api-access-h7tgz\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:33:11.951533 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:11.951500    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b813802b-398c-4cf0-84e1-17c63c1ea9d7-metrics-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:33:12.037514 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:12.037463    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:12.164494 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:12.164459    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:12.619153 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:12.619113    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"
Sep 10 17:33:12.619153 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:12.619145    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth" event={"ID":"b813802b-398c-4cf0-84e1-17c63c1ea9d7","Type":"ContainerDied","Data":"c814462c2ddba4dbf84a33cc2bfbdc6ad504b4509efb6a781c67d530d540d7e0"}
Sep 10 17:33:12.619647 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:12.619208    2566 scope.go:117] "RemoveContainer" containerID="a6380989311c0860b779fcc6d61b34436c95f106b8cee0df65241f1311356ffa"
Sep 10 17:33:12.642374 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:12.642349    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"]
Sep 10 17:33:12.646569 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:12.646543    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-851d1193-6bbdf665f-69fth"]
Sep 10 17:33:12.834882 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:12.834848    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f5f39e1-6f18-48c8-98ef-cd276ab331f8" path="/var/lib/kubelet/pods/3f5f39e1-6f18-48c8-98ef-cd276ab331f8/volumes"
Sep 10 17:33:12.835271 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:12.835257    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b813802b-398c-4cf0-84e1-17c63c1ea9d7" path="/var/lib/kubelet/pods/b813802b-398c-4cf0-84e1-17c63c1ea9d7/volumes"
Sep 10 17:33:12.891740 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:12.891655    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:13.862811 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:13.862773    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:15.616869 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:15.616837    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:16.035086 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:16.035048    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:16.603861 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:16.603824    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:16.726546 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:16.726504    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:16.952773 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:16.952680    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:17.071849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:17.071816    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:17.709829 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:17.709776    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:17.813926 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:17.813883    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8c46e8-kdq42"]
Sep 10 17:33:17.814351 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:17.814337    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b813802b-398c-4cf0-84e1-17c63c1ea9d7" containerName="maas-api"
Sep 10 17:33:17.814351 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:17.814352    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b813802b-398c-4cf0-84e1-17c63c1ea9d7" containerName="maas-api"
Sep 10 17:33:17.814438 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:17.814368    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f5f39e1-6f18-48c8-98ef-cd276ab331f8" containerName="revoke-keys"
Sep 10 17:33:17.814438 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:17.814374    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f5f39e1-6f18-48c8-98ef-cd276ab331f8" containerName="revoke-keys"
Sep 10 17:33:17.814498 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:17.814443    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b813802b-398c-4cf0-84e1-17c63c1ea9d7" containerName="maas-api"
Sep 10 17:33:17.814498 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:17.814452    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f5f39e1-6f18-48c8-98ef-cd276ab331f8" containerName="revoke-keys"
Sep 10 17:33:17.818884 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:17.818867    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8c46e8-kdq42"
Sep 10 17:33:17.821618 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:17.821593    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxddn\""
Sep 10 17:33:17.821720 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:17.821676    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:33:17.825921 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:17.825900    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8c46e8-kdq42"]
Sep 10 17:33:18.012557 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:18.012514    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b64e24ef-1d24-4a83-9713-6b320d43d0eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8c46e8-kdq42\" (UID: \"b64e24ef-1d24-4a83-9713-6b320d43d0eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8c46e8-kdq42"
Sep 10 17:33:18.058050 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:18.058020    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:18.114030 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:18.113997    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b64e24ef-1d24-4a83-9713-6b320d43d0eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8c46e8-kdq42\" (UID: \"b64e24ef-1d24-4a83-9713-6b320d43d0eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8c46e8-kdq42"
Sep 10 17:33:18.114624 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:18.114602    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b64e24ef-1d24-4a83-9713-6b320d43d0eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8c46e8-kdq42\" (UID: \"b64e24ef-1d24-4a83-9713-6b320d43d0eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8c46e8-kdq42"
Sep 10 17:33:18.131982 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:18.131955    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8c46e8-kdq42"
Sep 10 17:33:18.158289 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:18.158243    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:18.328306 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:18.328216    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:18.410521 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:18.410071    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8c46e8-kdq42"]
Sep 10 17:33:18.410521 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:33:18.410473    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb64e24ef_1d24_4a83_9713_6b320d43d0eb.slice/crio-af1e9372693c3c0bc8c082e69f35c08edaeed0ce9aeafe3ae8b139d3562afb7e WatchSource:0}: Error finding container af1e9372693c3c0bc8c082e69f35c08edaeed0ce9aeafe3ae8b139d3562afb7e: Status 404 returned error can't find the container with id af1e9372693c3c0bc8c082e69f35c08edaeed0ce9aeafe3ae8b139d3562afb7e
Sep 10 17:33:18.656177 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:18.656147    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8c46e8-kdq42" event={"ID":"b64e24ef-1d24-4a83-9713-6b320d43d0eb","Type":"ContainerStarted","Data":"af1e9372693c3c0bc8c082e69f35c08edaeed0ce9aeafe3ae8b139d3562afb7e"}
Sep 10 17:33:18.692502 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:18.692472    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:18.789023 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:18.788984    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:18.902786 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:18.902702    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:19.663432 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:19.663345    2566 generic.go:356] "Generic (PLEG): container finished" podID="b64e24ef-1d24-4a83-9713-6b320d43d0eb" containerID="f4d83caec8b3afa24f625fe6d9f830c75b85d35b31be1d79c9060a2fd219d74a" exitCode=0
Sep 10 17:33:19.663578 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:19.663429    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8c46e8-kdq42" event={"ID":"b64e24ef-1d24-4a83-9713-6b320d43d0eb","Type":"ContainerDied","Data":"f4d83caec8b3afa24f625fe6d9f830c75b85d35b31be1d79c9060a2fd219d74a"}
Sep 10 17:33:19.879397 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:19.879363    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:20.804120 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:20.804093    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8c46e8-kdq42"
Sep 10 17:33:20.845158 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:20.845108    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b64e24ef-1d24-4a83-9713-6b320d43d0eb-maas-api-service-ca\") pod \"b64e24ef-1d24-4a83-9713-6b320d43d0eb\" (UID: \"b64e24ef-1d24-4a83-9713-6b320d43d0eb\") "
Sep 10 17:33:20.845429 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:20.845397    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b64e24ef-1d24-4a83-9713-6b320d43d0eb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b64e24ef-1d24-4a83-9713-6b320d43d0eb" (UID: "b64e24ef-1d24-4a83-9713-6b320d43d0eb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:33:20.892827 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:20.892796    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:20.946862 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:20.946773    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b64e24ef-1d24-4a83-9713-6b320d43d0eb-maas-api-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:33:21.677512 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:21.677480    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8c46e8-kdq42"
Sep 10 17:33:21.677680 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:21.677481    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8c46e8-kdq42" event={"ID":"b64e24ef-1d24-4a83-9713-6b320d43d0eb","Type":"ContainerDied","Data":"af1e9372693c3c0bc8c082e69f35c08edaeed0ce9aeafe3ae8b139d3562afb7e"}
Sep 10 17:33:21.677680 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:21.677605    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af1e9372693c3c0bc8c082e69f35c08edaeed0ce9aeafe3ae8b139d3562afb7e"
Sep 10 17:33:22.140427 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:22.140393    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:25.143459 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:25.143427    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:25.373425 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:25.373391    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:25.487416 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:25.487323    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:28.524621 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.524584    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:28.532188 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.532162    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8c46e8-29817690-bht6g"]
Sep 10 17:33:28.537633 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.537604    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8c46e8-29817690-bht6g"]
Sep 10 17:33:28.578225 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.578198    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p"]
Sep 10 17:33:28.578482 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.578453    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p" podUID="23988339-54db-4c05-b849-f669e303ab29" containerName="payload-pre-processing" containerID="cri-o://85dcd6cae7f9b930dafbc456a68788f875afeaf2af0e59f1d76a24aba526898e" gracePeriod=30
Sep 10 17:33:28.631388 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.631351    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:28.725671 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.725636    2566 generic.go:356] "Generic (PLEG): container finished" podID="23988339-54db-4c05-b849-f669e303ab29" containerID="85dcd6cae7f9b930dafbc456a68788f875afeaf2af0e59f1d76a24aba526898e" exitCode=0
Sep 10 17:33:28.726087 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.725694    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p" event={"ID":"23988339-54db-4c05-b849-f669e303ab29","Type":"ContainerDied","Data":"85dcd6cae7f9b930dafbc456a68788f875afeaf2af0e59f1d76a24aba526898e"}
Sep 10 17:33:28.730897 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.730870    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:28.780183 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.780093    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8c46e8-kdq42"]
Sep 10 17:33:28.784396 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.784364    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8c46e8-kdq42"]
Sep 10 17:33:28.809568 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.809513    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:28.835527 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.835502    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b64e24ef-1d24-4a83-9713-6b320d43d0eb" path="/var/lib/kubelet/pods/b64e24ef-1d24-4a83-9713-6b320d43d0eb/volumes"
Sep 10 17:33:28.835814 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.835800    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff19c3ea-8ee9-455d-9989-a5a607dc6cc4" path="/var/lib/kubelet/pods/ff19c3ea-8ee9-455d-9989-a5a607dc6cc4/volumes"
Sep 10 17:33:28.842483 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.842464    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p"
Sep 10 17:33:28.924669 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.924641    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bw4rn\" (UniqueName: \"kubernetes.io/projected/23988339-54db-4c05-b849-f669e303ab29-kube-api-access-bw4rn\") pod \"23988339-54db-4c05-b849-f669e303ab29\" (UID: \"23988339-54db-4c05-b849-f669e303ab29\") "
Sep 10 17:33:28.924830 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.924676    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23988339-54db-4c05-b849-f669e303ab29-plugins-config-volume\") pod \"23988339-54db-4c05-b849-f669e303ab29\" (UID: \"23988339-54db-4c05-b849-f669e303ab29\") "
Sep 10 17:33:28.925138 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.925101    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23988339-54db-4c05-b849-f669e303ab29-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "23988339-54db-4c05-b849-f669e303ab29" (UID: "23988339-54db-4c05-b849-f669e303ab29"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:33:28.926829 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:28.926803    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23988339-54db-4c05-b849-f669e303ab29-kube-api-access-bw4rn" (OuterVolumeSpecName: "kube-api-access-bw4rn") pod "23988339-54db-4c05-b849-f669e303ab29" (UID: "23988339-54db-4c05-b849-f669e303ab29"). InnerVolumeSpecName "kube-api-access-bw4rn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:33:29.025576 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:29.025541    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bw4rn\" (UniqueName: \"kubernetes.io/projected/23988339-54db-4c05-b849-f669e303ab29-kube-api-access-bw4rn\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:33:29.025576 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:29.025568    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23988339-54db-4c05-b849-f669e303ab29-plugins-config-volume\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:33:29.121182 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:29.121087    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:29.250912 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:29.250870    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:29.735352 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:29.735323    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p"
Sep 10 17:33:29.735352 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:29.735346    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p" event={"ID":"23988339-54db-4c05-b849-f669e303ab29","Type":"ContainerDied","Data":"b3c0c75ff73585fa1051500cda3b73cc4007c9cd45085e3901f31097de3136cd"}
Sep 10 17:33:29.735881 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:29.735403    2566 scope.go:117] "RemoveContainer" containerID="85dcd6cae7f9b930dafbc456a68788f875afeaf2af0e59f1d76a24aba526898e"
Sep 10 17:33:29.748878 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:33:29.748855    2566 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23988339_54db_4c05_b849_f669e303ab29.slice/crio-b3c0c75ff73585fa1051500cda3b73cc4007c9cd45085e3901f31097de3136cd\": RecentStats: unable to find data in memory cache]"
Sep 10 17:33:29.748976 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:33:29.748894    2566 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23988339_54db_4c05_b849_f669e303ab29.slice/crio-b3c0c75ff73585fa1051500cda3b73cc4007c9cd45085e3901f31097de3136cd\": RecentStats: unable to find data in memory cache]"
Sep 10 17:33:29.762966 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:29.762939    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p"]
Sep 10 17:33:29.765295 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:29.765271    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-8c46e8-5946bc4fdb-mbn9p"]
Sep 10 17:33:29.971486 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:29.971445    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:30.834558 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:30.834521    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23988339-54db-4c05-b849-f669e303ab29" path="/var/lib/kubelet/pods/23988339-54db-4c05-b849-f669e303ab29/volumes"
Sep 10 17:33:31.646097 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:31.646065    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:31.715549 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:31.715516    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:32.522956 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:32.522919    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:32.848188 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:32.848077    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:33.287416 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:33.287379    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:35.326489 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:35.326452    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:35.417050 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:35.417004    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:35.573863 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:35.573833    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:36.051866 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.051831    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:36.068915 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.068887    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-15ece2-8bq4h"]
Sep 10 17:33:36.069731 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.069697    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b64e24ef-1d24-4a83-9713-6b320d43d0eb" containerName="revoke-keys"
Sep 10 17:33:36.069731 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.069724    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b64e24ef-1d24-4a83-9713-6b320d43d0eb" containerName="revoke-keys"
Sep 10 17:33:36.069904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.069742    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23988339-54db-4c05-b849-f669e303ab29" containerName="payload-pre-processing"
Sep 10 17:33:36.069904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.069751    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="23988339-54db-4c05-b849-f669e303ab29" containerName="payload-pre-processing"
Sep 10 17:33:36.069904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.069854    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="23988339-54db-4c05-b849-f669e303ab29" containerName="payload-pre-processing"
Sep 10 17:33:36.069904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.069871    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b64e24ef-1d24-4a83-9713-6b320d43d0eb" containerName="revoke-keys"
Sep 10 17:33:36.074729 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.074708    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-15ece2-8bq4h"
Sep 10 17:33:36.086059 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.078334    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:33:36.086059 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.078382    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxddn\""
Sep 10 17:33:36.089309 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.089285    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-15ece2-8bq4h"]
Sep 10 17:33:36.136108 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.136072    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:36.197542 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.197502    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10492627-4c39-4c72-96f6-957108a4f09c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-15ece2-8bq4h\" (UID: \"10492627-4c39-4c72-96f6-957108a4f09c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-15ece2-8bq4h"
Sep 10 17:33:36.298436 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.298400    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10492627-4c39-4c72-96f6-957108a4f09c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-15ece2-8bq4h\" (UID: \"10492627-4c39-4c72-96f6-957108a4f09c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-15ece2-8bq4h"
Sep 10 17:33:36.299003 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.298983    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10492627-4c39-4c72-96f6-957108a4f09c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-15ece2-8bq4h\" (UID: \"10492627-4c39-4c72-96f6-957108a4f09c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-15ece2-8bq4h"
Sep 10 17:33:36.394930 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.394854    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-15ece2-8bq4h"
Sep 10 17:33:36.674111 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.674081    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-15ece2-8bq4h"]
Sep 10 17:33:36.676146 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:33:36.676100    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10492627_4c39_4c72_96f6_957108a4f09c.slice/crio-2e5c3026958d4b06663dda96654de74b4f2e797e06a52282dc17cbc5c05af70c WatchSource:0}: Error finding container 2e5c3026958d4b06663dda96654de74b4f2e797e06a52282dc17cbc5c05af70c: Status 404 returned error can't find the container with id 2e5c3026958d4b06663dda96654de74b4f2e797e06a52282dc17cbc5c05af70c
Sep 10 17:33:36.779147 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:36.779104    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-15ece2-8bq4h" event={"ID":"10492627-4c39-4c72-96f6-957108a4f09c","Type":"ContainerStarted","Data":"2e5c3026958d4b06663dda96654de74b4f2e797e06a52282dc17cbc5c05af70c"}
Sep 10 17:33:37.265476 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:37.265435    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:37.788327 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:37.788294    2566 generic.go:356] "Generic (PLEG): container finished" podID="10492627-4c39-4c72-96f6-957108a4f09c" containerID="a54f6402f4ed94d705cbbd28ee93974965f39f43e049f6658b923c61ccda3f82" exitCode=0
Sep 10 17:33:37.788774 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:37.788334    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-15ece2-8bq4h" event={"ID":"10492627-4c39-4c72-96f6-957108a4f09c","Type":"ContainerDied","Data":"a54f6402f4ed94d705cbbd28ee93974965f39f43e049f6658b923c61ccda3f82"}
Sep 10 17:33:38.206620 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:38.206535    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:38.926317 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:38.926292    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-15ece2-8bq4h"
Sep 10 17:33:39.025039 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.025009    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10492627-4c39-4c72-96f6-957108a4f09c-maas-api-service-ca\") pod \"10492627-4c39-4c72-96f6-957108a4f09c\" (UID: \"10492627-4c39-4c72-96f6-957108a4f09c\") "
Sep 10 17:33:39.025449 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.025425    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10492627-4c39-4c72-96f6-957108a4f09c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "10492627-4c39-4c72-96f6-957108a4f09c" (UID: "10492627-4c39-4c72-96f6-957108a4f09c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:33:39.033479 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.033456    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"]
Sep 10 17:33:39.033913 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.033900    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10492627-4c39-4c72-96f6-957108a4f09c" containerName="revoke-keys"
Sep 10 17:33:39.033959 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.033915    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="10492627-4c39-4c72-96f6-957108a4f09c" containerName="revoke-keys"
Sep 10 17:33:39.034005 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.033987    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="10492627-4c39-4c72-96f6-957108a4f09c" containerName="revoke-keys"
Sep 10 17:33:39.037084 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.037064    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:33:39.039775 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.039754    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-2hfkp\""
Sep 10 17:33:39.039909 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.039794    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 17:33:39.039909 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.039818    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 17:33:39.039909 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.039792    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-52f72f-serving-cert\""
Sep 10 17:33:39.050831 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.050809    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"]
Sep 10 17:33:39.062897 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.062839    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:39.126101 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.126066    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1170928a-f7e9-4b2a-bc57-106d01bdaf88-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq\" (UID: \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:33:39.126318 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.126194    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1170928a-f7e9-4b2a-bc57-106d01bdaf88-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq\" (UID: \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:33:39.126318 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.126283    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-499r9\" (UniqueName: \"kubernetes.io/projected/1170928a-f7e9-4b2a-bc57-106d01bdaf88-kube-api-access-499r9\") pod \"maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq\" (UID: \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:33:39.126450 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.126331    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1170928a-f7e9-4b2a-bc57-106d01bdaf88-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq\" (UID: \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:33:39.126450 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.126425    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10492627-4c39-4c72-96f6-957108a4f09c-maas-api-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:33:39.130091 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.130067    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:39.227776 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.227741    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-499r9\" (UniqueName: \"kubernetes.io/projected/1170928a-f7e9-4b2a-bc57-106d01bdaf88-kube-api-access-499r9\") pod \"maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq\" (UID: \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:33:39.227959 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.227793    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1170928a-f7e9-4b2a-bc57-106d01bdaf88-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq\" (UID: \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:33:39.227959 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.227880    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1170928a-f7e9-4b2a-bc57-106d01bdaf88-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq\" (UID: \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:33:39.227959 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.227949    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1170928a-f7e9-4b2a-bc57-106d01bdaf88-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq\" (UID: \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:33:39.228120 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:33:39.228094    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-serving-cert: secret "maas-api-e2e-shared-a-w2-52f72f-serving-cert" not found
Sep 10 17:33:39.228198 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:33:39.228183    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1170928a-f7e9-4b2a-bc57-106d01bdaf88-maas-api-tls podName:1170928a-f7e9-4b2a-bc57-106d01bdaf88 nodeName:}" failed. No retries permitted until 2026-09-10 17:33:39.728162685 +0000 UTC m=+1223.409335611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1170928a-f7e9-4b2a-bc57-106d01bdaf88-maas-api-tls") pod "maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq" (UID: "1170928a-f7e9-4b2a-bc57-106d01bdaf88") : secret "maas-api-e2e-shared-a-w2-52f72f-serving-cert" not found
Sep 10 17:33:39.228622 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.228598    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1170928a-f7e9-4b2a-bc57-106d01bdaf88-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq\" (UID: \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:33:39.230375 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.230355    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1170928a-f7e9-4b2a-bc57-106d01bdaf88-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq\" (UID: \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:33:39.239389 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.239365    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-499r9\" (UniqueName: \"kubernetes.io/projected/1170928a-f7e9-4b2a-bc57-106d01bdaf88-kube-api-access-499r9\") pod \"maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq\" (UID: \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:33:39.665179 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.665116    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:39.733172 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.733138    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1170928a-f7e9-4b2a-bc57-106d01bdaf88-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq\" (UID: \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:33:39.733348 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:33:39.733278    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-serving-cert: secret "maas-api-e2e-shared-a-w2-52f72f-serving-cert" not found
Sep 10 17:33:39.733348 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:33:39.733337    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1170928a-f7e9-4b2a-bc57-106d01bdaf88-maas-api-tls podName:1170928a-f7e9-4b2a-bc57-106d01bdaf88 nodeName:}" failed. No retries permitted until 2026-09-10 17:33:40.733321732 +0000 UTC m=+1224.414494652 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1170928a-f7e9-4b2a-bc57-106d01bdaf88-maas-api-tls") pod "maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq" (UID: "1170928a-f7e9-4b2a-bc57-106d01bdaf88") : secret "maas-api-e2e-shared-a-w2-52f72f-serving-cert" not found
Sep 10 17:33:39.765189 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.765114    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:39.804870 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.804844    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-15ece2-8bq4h"
Sep 10 17:33:39.805010 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.804887    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-15ece2-8bq4h" event={"ID":"10492627-4c39-4c72-96f6-957108a4f09c","Type":"ContainerDied","Data":"2e5c3026958d4b06663dda96654de74b4f2e797e06a52282dc17cbc5c05af70c"}
Sep 10 17:33:39.805010 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.804930    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e5c3026958d4b06663dda96654de74b4f2e797e06a52282dc17cbc5c05af70c"
Sep 10 17:33:39.914979 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:39.914947    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:40.743422 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:40.743387    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1170928a-f7e9-4b2a-bc57-106d01bdaf88-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq\" (UID: \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:33:40.745848 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:40.745824    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1170928a-f7e9-4b2a-bc57-106d01bdaf88-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq\" (UID: \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:33:40.850494 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:40.850463    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:33:41.130170 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:41.130142    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"]
Sep 10 17:33:41.131799 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:33:41.131770    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1170928a_f7e9_4b2a_bc57_106d01bdaf88.slice/crio-3dbdb74490b126e6099ea129fb9fd88e74cf2d2d3134e35a9adff21461ddbec0 WatchSource:0}: Error finding container 3dbdb74490b126e6099ea129fb9fd88e74cf2d2d3134e35a9adff21461ddbec0: Status 404 returned error can't find the container with id 3dbdb74490b126e6099ea129fb9fd88e74cf2d2d3134e35a9adff21461ddbec0
Sep 10 17:33:41.818981 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:41.818943    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq" event={"ID":"1170928a-f7e9-4b2a-bc57-106d01bdaf88","Type":"ContainerStarted","Data":"479c6c6e707d5df003e8910fecf6cac479bd26b19a0a5a70a040c804f8fc805e"}
Sep 10 17:33:41.819369 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:41.819012    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:33:41.819369 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:41.819036    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq" event={"ID":"1170928a-f7e9-4b2a-bc57-106d01bdaf88","Type":"ContainerStarted","Data":"3dbdb74490b126e6099ea129fb9fd88e74cf2d2d3134e35a9adff21461ddbec0"}
Sep 10 17:33:41.837776 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:41.837686    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq" podStartSLOduration=2.393438852 podStartE2EDuration="2.837670702s" totalImagesPullingTime="444.23185ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:33:41.836452847 +0000 UTC m=+1225.517625789" watchObservedRunningTime="2026-09-10 17:33:41.837670702 +0000 UTC m=+1225.518843645"
Sep 10 17:33:46.224862 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:46.224828    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:47.291013 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:47.290974    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:47.431229 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:47.431191    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:47.539000 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:47.538966    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:47.658321 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:47.658232    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:47.733965 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:47.733929    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-15ece2-8bq4h"]
Sep 10 17:33:47.736735 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:47.736712    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-15ece2-8bq4h"]
Sep 10 17:33:47.798775 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:47.798745    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:47.831383 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:47.831356    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:33:48.109316 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:48.109276    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:48.369872 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:48.369755    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:48.835145 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:48.835086    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10492627-4c39-4c72-96f6-957108a4f09c" path="/var/lib/kubelet/pods/10492627-4c39-4c72-96f6-957108a4f09c/volumes"
Sep 10 17:33:49.495111 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:49.495077    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:51.537296 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:51.537256    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:52.678191 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:52.678150    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:52.983142 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:52.983035    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:53.080810 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:53.080769    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:54.566424 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:54.566391    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:54.664743 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:54.664708    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:54.792452 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:54.792415    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"]
Sep 10 17:33:54.792760 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:54.792735    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw" podUID="e0bd13d0-885d-4044-8425-6fe48e338199" containerName="istio-proxy" containerID="cri-o://9b4ca983e632cc4b7d49225787e09c335097137232ebae6e328f902265241f5f" gracePeriod=30
Sep 10 17:33:54.811578 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:54.811548    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:55.343397 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:55.343365    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:55.721272 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:55.721185    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:55.823683 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:55.823649    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:56.060210 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:56.060171    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:56.157896 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:56.157856    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:56.413525 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:56.413437    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:56.607429 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:56.607394    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:58.541289 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:58.541233    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:59.197686 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:59.197628    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:59.295182 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:59.295103    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:59.730661 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:59.730625    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:59.879579 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:59.879542    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:33:59.931882 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:59.931849    2566 generic.go:356] "Generic (PLEG): container finished" podID="e0bd13d0-885d-4044-8425-6fe48e338199" containerID="9b4ca983e632cc4b7d49225787e09c335097137232ebae6e328f902265241f5f" exitCode=0
Sep 10 17:33:59.932078 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:59.931883    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw" event={"ID":"e0bd13d0-885d-4044-8425-6fe48e338199","Type":"ContainerDied","Data":"9b4ca983e632cc4b7d49225787e09c335097137232ebae6e328f902265241f5f"}
Sep 10 17:33:59.932078 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:33:59.931948    2566 scope.go:117] "RemoveContainer" containerID="167a8b2e4b78ef234b7c1c477ca74bcc0f32f78450640e85c044dfc9f281fc72"
Sep 10 17:34:00.041745 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.041719    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:34:00.132557 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.132525    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-credential-socket\") pod \"e0bd13d0-885d-4044-8425-6fe48e338199\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") "
Sep 10 17:34:00.132728 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.132565    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-istio-envoy\") pod \"e0bd13d0-885d-4044-8425-6fe48e338199\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") "
Sep 10 17:34:00.132728 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.132601    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0bd13d0-885d-4044-8425-6fe48e338199-istio-podinfo\") pod \"e0bd13d0-885d-4044-8425-6fe48e338199\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") "
Sep 10 17:34:00.132728 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.132623    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0bd13d0-885d-4044-8425-6fe48e338199-istio-token\") pod \"e0bd13d0-885d-4044-8425-6fe48e338199\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") "
Sep 10 17:34:00.132728 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.132664    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0bd13d0-885d-4044-8425-6fe48e338199-istiod-ca-cert\") pod \"e0bd13d0-885d-4044-8425-6fe48e338199\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") "
Sep 10 17:34:00.132728 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.132687    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-workload-socket\") pod \"e0bd13d0-885d-4044-8425-6fe48e338199\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") "
Sep 10 17:34:00.132728 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.132718    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-istio-data\") pod \"e0bd13d0-885d-4044-8425-6fe48e338199\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") "
Sep 10 17:34:00.133048 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.132737    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfm9x\" (UniqueName: \"kubernetes.io/projected/e0bd13d0-885d-4044-8425-6fe48e338199-kube-api-access-kfm9x\") pod \"e0bd13d0-885d-4044-8425-6fe48e338199\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") "
Sep 10 17:34:00.133048 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.132837    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-workload-certs\") pod \"e0bd13d0-885d-4044-8425-6fe48e338199\" (UID: \"e0bd13d0-885d-4044-8425-6fe48e338199\") "
Sep 10 17:34:00.133197 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.132829    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e0bd13d0-885d-4044-8425-6fe48e338199" (UID: "e0bd13d0-885d-4044-8425-6fe48e338199"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:34:00.133197 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.133060    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-istio-data" (OuterVolumeSpecName: "istio-data") pod "e0bd13d0-885d-4044-8425-6fe48e338199" (UID: "e0bd13d0-885d-4044-8425-6fe48e338199"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:34:00.133197 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.133082    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0bd13d0-885d-4044-8425-6fe48e338199-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e0bd13d0-885d-4044-8425-6fe48e338199" (UID: "e0bd13d0-885d-4044-8425-6fe48e338199"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:34:00.133334 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.133234    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e0bd13d0-885d-4044-8425-6fe48e338199" (UID: "e0bd13d0-885d-4044-8425-6fe48e338199"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:34:00.133334 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.133272    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e0bd13d0-885d-4044-8425-6fe48e338199" (UID: "e0bd13d0-885d-4044-8425-6fe48e338199"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:34:00.134808 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.134776    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e0bd13d0-885d-4044-8425-6fe48e338199-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e0bd13d0-885d-4044-8425-6fe48e338199" (UID: "e0bd13d0-885d-4044-8425-6fe48e338199"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 17:34:00.134962 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.134913    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0bd13d0-885d-4044-8425-6fe48e338199-kube-api-access-kfm9x" (OuterVolumeSpecName: "kube-api-access-kfm9x") pod "e0bd13d0-885d-4044-8425-6fe48e338199" (UID: "e0bd13d0-885d-4044-8425-6fe48e338199"). InnerVolumeSpecName "kube-api-access-kfm9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:34:00.135241 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.135217    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e0bd13d0-885d-4044-8425-6fe48e338199" (UID: "e0bd13d0-885d-4044-8425-6fe48e338199"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:34:00.135319 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.135214    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0bd13d0-885d-4044-8425-6fe48e338199-istio-token" (OuterVolumeSpecName: "istio-token") pod "e0bd13d0-885d-4044-8425-6fe48e338199" (UID: "e0bd13d0-885d-4044-8425-6fe48e338199"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:34:00.234194 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.234155    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-credential-socket\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:34:00.234194 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.234188    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-istio-envoy\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:34:00.234194 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.234201    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0bd13d0-885d-4044-8425-6fe48e338199-istio-podinfo\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:34:00.234478 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.234212    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0bd13d0-885d-4044-8425-6fe48e338199-istio-token\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:34:00.234478 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.234223    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0bd13d0-885d-4044-8425-6fe48e338199-istiod-ca-cert\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:34:00.234478 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.234234    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-workload-socket\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:34:00.234478 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.234244    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-istio-data\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:34:00.234478 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.234255    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfm9x\" (UniqueName: \"kubernetes.io/projected/e0bd13d0-885d-4044-8425-6fe48e338199-kube-api-access-kfm9x\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:34:00.234478 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.234268    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0bd13d0-885d-4044-8425-6fe48e338199-workload-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:34:00.943385 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.943348    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw" event={"ID":"e0bd13d0-885d-4044-8425-6fe48e338199","Type":"ContainerDied","Data":"6170a7c61d1c05b52fd5beb458daebd88ceeede190d93633cb6aa114aebdd73c"}
Sep 10 17:34:00.943385 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.943365    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"
Sep 10 17:34:00.943900 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.943414    2566 scope.go:117] "RemoveContainer" containerID="9b4ca983e632cc4b7d49225787e09c335097137232ebae6e328f902265241f5f"
Sep 10 17:34:00.980880 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.980849    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"]
Sep 10 17:34:00.987113 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:00.987086    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-15ece2-openshift-default-55f4dd995d-skhdw"]
Sep 10 17:34:01.087822 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.087790    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"]
Sep 10 17:34:01.088233 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.088219    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0bd13d0-885d-4044-8425-6fe48e338199" containerName="istio-proxy"
Sep 10 17:34:01.088282 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.088234    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0bd13d0-885d-4044-8425-6fe48e338199" containerName="istio-proxy"
Sep 10 17:34:01.088357 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.088346    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0bd13d0-885d-4044-8425-6fe48e338199" containerName="istio-proxy"
Sep 10 17:34:01.092793 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.092731    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"
Sep 10 17:34:01.105696 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.105660    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-5912a3-serving-cert\""
Sep 10 17:34:01.131147 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.131022    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"]
Sep 10 17:34:01.143326 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.143299    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj\" (UID: \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"
Sep 10 17:34:01.143458 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.143344    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhkcg\" (UniqueName: \"kubernetes.io/projected/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-kube-api-access-fhkcg\") pod \"maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj\" (UID: \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"
Sep 10 17:34:01.143458 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.143391    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj\" (UID: \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"
Sep 10 17:34:01.143549 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.143511    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj\" (UID: \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"
Sep 10 17:34:01.186528 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.186491    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:01.244830 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.244795    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj\" (UID: \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"
Sep 10 17:34:01.245002 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.244838    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhkcg\" (UniqueName: \"kubernetes.io/projected/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-kube-api-access-fhkcg\") pod \"maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj\" (UID: \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"
Sep 10 17:34:01.245002 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.244863    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj\" (UID: \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"
Sep 10 17:34:01.245002 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.244926    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj\" (UID: \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"
Sep 10 17:34:01.245677 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.245652    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj\" (UID: \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"
Sep 10 17:34:01.247307 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.247287    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj\" (UID: \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"
Sep 10 17:34:01.247430 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.247410    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj\" (UID: \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"
Sep 10 17:34:01.259980 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.259959    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhkcg\" (UniqueName: \"kubernetes.io/projected/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-kube-api-access-fhkcg\") pod \"maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj\" (UID: \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"
Sep 10 17:34:01.305898 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.305866    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:01.404155 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.404055    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"
Sep 10 17:34:01.690092 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:34:01.690066    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9ba7f62_2ff3_4263_a81e_c4ddc8e72870.slice/crio-cfd94ed6d55bec37f0c59a1e083f90f0510624e2a6b3bf1d2ae16531ae648120 WatchSource:0}: Error finding container cfd94ed6d55bec37f0c59a1e083f90f0510624e2a6b3bf1d2ae16531ae648120: Status 404 returned error can't find the container with id cfd94ed6d55bec37f0c59a1e083f90f0510624e2a6b3bf1d2ae16531ae648120
Sep 10 17:34:01.690322 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.690299    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"]
Sep 10 17:34:01.951248 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:01.951148    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj" event={"ID":"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870","Type":"ContainerStarted","Data":"cfd94ed6d55bec37f0c59a1e083f90f0510624e2a6b3bf1d2ae16531ae648120"}
Sep 10 17:34:02.017617 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:02.017595    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:02.265543 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:02.265514    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:02.788664 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:02.788635    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:02.840304 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:02.840273    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0bd13d0-885d-4044-8425-6fe48e338199" path="/var/lib/kubelet/pods/e0bd13d0-885d-4044-8425-6fe48e338199/volumes"
Sep 10 17:34:02.959615 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:02.959580    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj" event={"ID":"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870","Type":"ContainerStarted","Data":"ea70e68352ce5f0349a7280f3e4c2e0eed392b7d048ae0f61ca7e440cd381eb7"}
Sep 10 17:34:02.960055 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:02.959684    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"
Sep 10 17:34:02.979610 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:02.979584    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"]
Sep 10 17:34:02.979996 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:02.979984    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0bd13d0-885d-4044-8425-6fe48e338199" containerName="istio-proxy"
Sep 10 17:34:02.980043 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:02.979998    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0bd13d0-885d-4044-8425-6fe48e338199" containerName="istio-proxy"
Sep 10 17:34:02.980091 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:02.980082    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0bd13d0-885d-4044-8425-6fe48e338199" containerName="istio-proxy"
Sep 10 17:34:02.983453 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:02.983437    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"
Sep 10 17:34:02.988452 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:02.988433    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-3c30a2-serving-cert\""
Sep 10 17:34:03.023736 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.023670    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj" podStartSLOduration=1.7211984340000002 podStartE2EDuration="2.023653752s" totalImagesPullingTime="302.455318ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:34:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:34:03.021560134 +0000 UTC m=+1246.702733099" watchObservedRunningTime="2026-09-10 17:34:03.023653752 +0000 UTC m=+1246.704826699"
Sep 10 17:34:03.027934 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.027907    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"]
Sep 10 17:34:03.049244 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.049057    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:03.061913 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.061886    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slsxg\" (UniqueName: \"kubernetes.io/projected/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-kube-api-access-slsxg\") pod \"maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn\" (UID: \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"
Sep 10 17:34:03.062049 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.061920    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn\" (UID: \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"
Sep 10 17:34:03.062049 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.061942    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn\" (UID: \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"
Sep 10 17:34:03.062147 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.062051    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn\" (UID: \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"
Sep 10 17:34:03.162965 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.162930    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn\" (UID: \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"
Sep 10 17:34:03.163182 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.163025    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slsxg\" (UniqueName: \"kubernetes.io/projected/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-kube-api-access-slsxg\") pod \"maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn\" (UID: \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"
Sep 10 17:34:03.163182 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.163059    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn\" (UID: \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"
Sep 10 17:34:03.163182 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.163083    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn\" (UID: \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"
Sep 10 17:34:03.163810 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.163784    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn\" (UID: \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"
Sep 10 17:34:03.165509 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.165484    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn\" (UID: \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"
Sep 10 17:34:03.165613 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.165592    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn\" (UID: \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"
Sep 10 17:34:03.191182 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.191161    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slsxg\" (UniqueName: \"kubernetes.io/projected/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-kube-api-access-slsxg\") pod \"maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn\" (UID: \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"
Sep 10 17:34:03.293619 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.293582    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:03.294396 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.294375    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"
Sep 10 17:34:03.587531 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.582100    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"]
Sep 10 17:34:03.969569 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:03.969535    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn" event={"ID":"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b","Type":"ContainerStarted","Data":"085da2ae86c7a1b906ddac1ecc700911622234a8b87a0ca46f3a6ea8e681d8a3"}
Sep 10 17:34:04.111206 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:04.111172    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:04.222151 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:04.222033    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:04.846068 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:04.846037    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:04.977260 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:04.977219    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn" event={"ID":"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b","Type":"ContainerStarted","Data":"f625fad7914ad08086aa18c732fa306a5b33c18db61104b2b34f11a141a32ea7"}
Sep 10 17:34:04.977639 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:04.977286    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"
Sep 10 17:34:05.019823 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:05.019771    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn" podStartSLOduration=2.6923716349999998 podStartE2EDuration="3.019754021s" totalImagesPullingTime="327.382386ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:34:05.016362103 +0000 UTC m=+1248.697535046" watchObservedRunningTime="2026-09-10 17:34:05.019754021 +0000 UTC m=+1248.700926964"
Sep 10 17:34:06.195289 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:06.195261    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:07.067105 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:07.067066    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:08.011265 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:08.011227    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:08.975417 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:08.975346    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"
Sep 10 17:34:10.990037 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:10.990004    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"
Sep 10 17:34:22.843198 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:22.843168    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:24.556964 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:24.556926    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:24.699609 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:24.699571    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:25.475435 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:25.475402    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7914-af378e"]
Sep 10 17:34:25.480446 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:25.480426    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7914-af378e"
Sep 10 17:34:25.483768 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:25.483731    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4s82r\""
Sep 10 17:34:25.485973 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:25.485947    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7914-af378e"]
Sep 10 17:34:25.576261 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:25.576228    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jl4t\" (UniqueName: \"kubernetes.io/projected/3f951752-d2a7-424c-a32d-b356503cd66d-kube-api-access-8jl4t\") pod \"test-curl-7914-af378e\" (UID: \"3f951752-d2a7-424c-a32d-b356503cd66d\") " pod="opendatahub/test-curl-7914-af378e"
Sep 10 17:34:25.677182 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:25.677151    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jl4t\" (UniqueName: \"kubernetes.io/projected/3f951752-d2a7-424c-a32d-b356503cd66d-kube-api-access-8jl4t\") pod \"test-curl-7914-af378e\" (UID: \"3f951752-d2a7-424c-a32d-b356503cd66d\") " pod="opendatahub/test-curl-7914-af378e"
Sep 10 17:34:25.686346 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:25.686312    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jl4t\" (UniqueName: \"kubernetes.io/projected/3f951752-d2a7-424c-a32d-b356503cd66d-kube-api-access-8jl4t\") pod \"test-curl-7914-af378e\" (UID: \"3f951752-d2a7-424c-a32d-b356503cd66d\") " pod="opendatahub/test-curl-7914-af378e"
Sep 10 17:34:25.725826 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:25.725752    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:25.791537 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:25.791493    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7914-af378e"
Sep 10 17:34:26.063826 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:26.063793    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7914-af378e"]
Sep 10 17:34:26.064409 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:34:26.064383    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f951752_d2a7_424c_a32d_b356503cd66d.slice/crio-f2fcaf25ecc91635cd6cec8d518b79fd499cf954494ff394e740499d849b5b79 WatchSource:0}: Error finding container f2fcaf25ecc91635cd6cec8d518b79fd499cf954494ff394e740499d849b5b79: Status 404 returned error can't find the container with id f2fcaf25ecc91635cd6cec8d518b79fd499cf954494ff394e740499d849b5b79
Sep 10 17:34:26.107925 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:26.107895    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7914-af378e" event={"ID":"3f951752-d2a7-424c-a32d-b356503cd66d","Type":"ContainerStarted","Data":"f2fcaf25ecc91635cd6cec8d518b79fd499cf954494ff394e740499d849b5b79"}
Sep 10 17:34:26.702594 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:26.702556    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:27.121051 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:27.121012    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7914-af378e" event={"ID":"3f951752-d2a7-424c-a32d-b356503cd66d","Type":"ContainerStarted","Data":"71f2c1ce82811e7d40f21d15f13b028c6b443af7033d7dca7eef377f103b1232"}
Sep 10 17:34:27.140906 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:27.140858    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7914-af378e" podStartSLOduration=2.140844056 podStartE2EDuration="2.140844056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:34:27.138565263 +0000 UTC m=+1270.819738218" watchObservedRunningTime="2026-09-10 17:34:27.140844056 +0000 UTC m=+1270.822016998"
Sep 10 17:34:27.699910 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:27.699871    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:27.852700 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:27.852650    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7914-af378e"]
Sep 10 17:34:27.856175 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:27.856080    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7914-af378e"]
Sep 10 17:34:27.994781 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:27.994745    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7914-09d9d3"]
Sep 10 17:34:27.995227 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:27.995213    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f951752-d2a7-424c-a32d-b356503cd66d" containerName="test-curl-7914-af378e"
Sep 10 17:34:27.995282 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:27.995228    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f951752-d2a7-424c-a32d-b356503cd66d" containerName="test-curl-7914-af378e"
Sep 10 17:34:27.995334 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:27.995325    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f951752-d2a7-424c-a32d-b356503cd66d" containerName="test-curl-7914-af378e"
Sep 10 17:34:27.998445 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:27.998430    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7914-09d9d3"
Sep 10 17:34:28.011864 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:28.011835    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7914-09d9d3"]
Sep 10 17:34:28.100553 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:28.100518    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5fmb\" (UniqueName: \"kubernetes.io/projected/f64beed9-c22c-446e-8134-4a93b0641181-kube-api-access-x5fmb\") pod \"test-curl-7914-09d9d3\" (UID: \"f64beed9-c22c-446e-8134-4a93b0641181\") " pod="opendatahub/test-curl-7914-09d9d3"
Sep 10 17:34:28.127453 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:34:28.127369    2566 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7914-af378e" container="test-curl-7914-af378e"
Sep 10 17:34:28.201587 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:28.201552    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5fmb\" (UniqueName: \"kubernetes.io/projected/f64beed9-c22c-446e-8134-4a93b0641181-kube-api-access-x5fmb\") pod \"test-curl-7914-09d9d3\" (UID: \"f64beed9-c22c-446e-8134-4a93b0641181\") " pod="opendatahub/test-curl-7914-09d9d3"
Sep 10 17:34:28.212118 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:28.212086    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5fmb\" (UniqueName: \"kubernetes.io/projected/f64beed9-c22c-446e-8134-4a93b0641181-kube-api-access-x5fmb\") pod \"test-curl-7914-09d9d3\" (UID: \"f64beed9-c22c-446e-8134-4a93b0641181\") " pod="opendatahub/test-curl-7914-09d9d3"
Sep 10 17:34:28.307932 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:28.307896    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7914-09d9d3"
Sep 10 17:34:28.583274 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:28.583245    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7914-09d9d3"]
Sep 10 17:34:28.585359 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:34:28.585331    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf64beed9_c22c_446e_8134_4a93b0641181.slice/crio-827c1a6a2e35ba3bfc6c39758355d834db41702c53fe7c09ea53c63c75238476 WatchSource:0}: Error finding container 827c1a6a2e35ba3bfc6c39758355d834db41702c53fe7c09ea53c63c75238476: Status 404 returned error can't find the container with id 827c1a6a2e35ba3bfc6c39758355d834db41702c53fe7c09ea53c63c75238476
Sep 10 17:34:28.656591 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:28.656557    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:29.135121 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:29.135085    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7914-09d9d3" event={"ID":"f64beed9-c22c-446e-8134-4a93b0641181","Type":"ContainerStarted","Data":"09558590ef78aa705d4ff67247ce67f897d524f91fa02c9126044f2613688f63"}
Sep 10 17:34:29.135545 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:29.135140    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7914-09d9d3" event={"ID":"f64beed9-c22c-446e-8134-4a93b0641181","Type":"ContainerStarted","Data":"827c1a6a2e35ba3bfc6c39758355d834db41702c53fe7c09ea53c63c75238476"}
Sep 10 17:34:29.135545 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:29.135214    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7914-af378e" podUID="3f951752-d2a7-424c-a32d-b356503cd66d" containerName="test-curl-7914-af378e" containerID="cri-o://71f2c1ce82811e7d40f21d15f13b028c6b443af7033d7dca7eef377f103b1232" gracePeriod=2
Sep 10 17:34:29.163387 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:29.163345    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7914-09d9d3" podStartSLOduration=2.163330095 podStartE2EDuration="2.163330095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:34:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:34:29.162359991 +0000 UTC m=+1272.843532934" watchObservedRunningTime="2026-09-10 17:34:29.163330095 +0000 UTC m=+1272.844503037"
Sep 10 17:34:29.219107 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:29.219077    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:29.573649 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:29.573610    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:29.707089 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:29.707054    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:29.818144 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:29.818090    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7914-09d9d3"]
Sep 10 17:34:29.822066 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:29.822038    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7914-09d9d3"]
Sep 10 17:34:29.938740 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:29.938663    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:29.962745 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:29.962709    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7914-04ee92"]
Sep 10 17:34:29.963568 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:29.963546    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f64beed9-c22c-446e-8134-4a93b0641181" containerName="test-curl-7914-09d9d3"
Sep 10 17:34:29.963568 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:29.963568    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f64beed9-c22c-446e-8134-4a93b0641181" containerName="test-curl-7914-09d9d3"
Sep 10 17:34:29.963737 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:29.963726    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f64beed9-c22c-446e-8134-4a93b0641181" containerName="test-curl-7914-09d9d3"
Sep 10 17:34:29.967544 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:29.967522    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7914-04ee92"
Sep 10 17:34:29.982368 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:29.982340    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7914-04ee92"]
Sep 10 17:34:30.121151 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:30.121100    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds9br\" (UniqueName: \"kubernetes.io/projected/a4110962-fff5-4e72-9739-bdc221196c30-kube-api-access-ds9br\") pod \"test-curl-7914-04ee92\" (UID: \"a4110962-fff5-4e72-9739-bdc221196c30\") " pod="opendatahub/test-curl-7914-04ee92"
Sep 10 17:34:30.142680 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:34:30.142649    2566 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7914-09d9d3" container="test-curl-7914-09d9d3"
Sep 10 17:34:30.222947 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:30.222843    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds9br\" (UniqueName: \"kubernetes.io/projected/a4110962-fff5-4e72-9739-bdc221196c30-kube-api-access-ds9br\") pod \"test-curl-7914-04ee92\" (UID: \"a4110962-fff5-4e72-9739-bdc221196c30\") " pod="opendatahub/test-curl-7914-04ee92"
Sep 10 17:34:30.235452 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:30.235425    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds9br\" (UniqueName: \"kubernetes.io/projected/a4110962-fff5-4e72-9739-bdc221196c30-kube-api-access-ds9br\") pod \"test-curl-7914-04ee92\" (UID: \"a4110962-fff5-4e72-9739-bdc221196c30\") " pod="opendatahub/test-curl-7914-04ee92"
Sep 10 17:34:30.276477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:30.276452    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7914-04ee92"
Sep 10 17:34:30.277016 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:30.276983    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:30.554701 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:30.554678    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7914-04ee92"]
Sep 10 17:34:30.556670 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:34:30.556638    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4110962_fff5_4e72_9739_bdc221196c30.slice/crio-49dbaede4c5d0a771015194f3366e582797a6edcd0483b6a4cd7d1ad26333a5a WatchSource:0}: Error finding container 49dbaede4c5d0a771015194f3366e582797a6edcd0483b6a4cd7d1ad26333a5a: Status 404 returned error can't find the container with id 49dbaede4c5d0a771015194f3366e582797a6edcd0483b6a4cd7d1ad26333a5a
Sep 10 17:34:30.808196 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:30.808094    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:31.150774 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:31.150741    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7914-04ee92" event={"ID":"a4110962-fff5-4e72-9739-bdc221196c30","Type":"ContainerStarted","Data":"302f3d390d9ad3c52a2de1b05fe78ecccf0b45df5a9e03b834c4469343d362ec"}
Sep 10 17:34:31.151074 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:31.150791    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7914-04ee92" event={"ID":"a4110962-fff5-4e72-9739-bdc221196c30","Type":"ContainerStarted","Data":"49dbaede4c5d0a771015194f3366e582797a6edcd0483b6a4cd7d1ad26333a5a"}
Sep 10 17:34:31.151074 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:31.150798    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7914-09d9d3" podUID="f64beed9-c22c-446e-8134-4a93b0641181" containerName="test-curl-7914-09d9d3" containerID="cri-o://09558590ef78aa705d4ff67247ce67f897d524f91fa02c9126044f2613688f63" gracePeriod=2
Sep 10 17:34:31.311717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:31.311682    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7914-af378e"
Sep 10 17:34:31.434907 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:31.434829    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jl4t\" (UniqueName: \"kubernetes.io/projected/3f951752-d2a7-424c-a32d-b356503cd66d-kube-api-access-8jl4t\") pod \"3f951752-d2a7-424c-a32d-b356503cd66d\" (UID: \"3f951752-d2a7-424c-a32d-b356503cd66d\") "
Sep 10 17:34:31.437038 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:31.437010    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f951752-d2a7-424c-a32d-b356503cd66d-kube-api-access-8jl4t" (OuterVolumeSpecName: "kube-api-access-8jl4t") pod "3f951752-d2a7-424c-a32d-b356503cd66d" (UID: "3f951752-d2a7-424c-a32d-b356503cd66d"). InnerVolumeSpecName "kube-api-access-8jl4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:34:31.536437 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:31.536404    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jl4t\" (UniqueName: \"kubernetes.io/projected/3f951752-d2a7-424c-a32d-b356503cd66d-kube-api-access-8jl4t\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:34:31.669884 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:31.669851    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:31.869544 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:31.869486    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7914-04ee92" podStartSLOduration=2.8694681109999998 podStartE2EDuration="2.869468111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:34:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:34:31.174652616 +0000 UTC m=+1274.855825558" watchObservedRunningTime="2026-09-10 17:34:31.869468111 +0000 UTC m=+1275.550641054"
Sep 10 17:34:31.869895 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:31.869875    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7914-04ee92"]
Sep 10 17:34:31.873787 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:31.873760    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7914-04ee92"]
Sep 10 17:34:31.996700 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:31.996659    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7914-615e3a"]
Sep 10 17:34:31.997104 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:31.997088    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4110962-fff5-4e72-9739-bdc221196c30" containerName="test-curl-7914-04ee92"
Sep 10 17:34:31.997104 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:31.997103    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4110962-fff5-4e72-9739-bdc221196c30" containerName="test-curl-7914-04ee92"
Sep 10 17:34:31.997270 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:31.997198    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4110962-fff5-4e72-9739-bdc221196c30" containerName="test-curl-7914-04ee92"
Sep 10 17:34:32.000108 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:32.000084    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7914-615e3a"
Sep 10 17:34:32.013309 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:32.013285    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7914-615e3a"]
Sep 10 17:34:32.142377 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:32.142290    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l296d\" (UniqueName: \"kubernetes.io/projected/e721e31b-3d72-47a5-af8d-46fe9ab792d8-kube-api-access-l296d\") pod \"test-curl-7914-615e3a\" (UID: \"e721e31b-3d72-47a5-af8d-46fe9ab792d8\") " pod="opendatahub/test-curl-7914-615e3a"
Sep 10 17:34:32.160834 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:32.160798    2566 generic.go:356] "Generic (PLEG): container finished" podID="3f951752-d2a7-424c-a32d-b356503cd66d" containerID="71f2c1ce82811e7d40f21d15f13b028c6b443af7033d7dca7eef377f103b1232" exitCode=137
Sep 10 17:34:32.161261 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:32.160843    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7914-af378e"
Sep 10 17:34:32.161261 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:32.160900    2566 scope.go:117] "RemoveContainer" containerID="71f2c1ce82811e7d40f21d15f13b028c6b443af7033d7dca7eef377f103b1232"
Sep 10 17:34:32.161261 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:34:32.161216    2566 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7914-04ee92" container="test-curl-7914-04ee92"
Sep 10 17:34:32.170635 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:32.170616    2566 scope.go:117] "RemoveContainer" containerID="71f2c1ce82811e7d40f21d15f13b028c6b443af7033d7dca7eef377f103b1232"
Sep 10 17:34:32.170935 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:34:32.170913    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71f2c1ce82811e7d40f21d15f13b028c6b443af7033d7dca7eef377f103b1232\": container with ID starting with 71f2c1ce82811e7d40f21d15f13b028c6b443af7033d7dca7eef377f103b1232 not found: ID does not exist" containerID="71f2c1ce82811e7d40f21d15f13b028c6b443af7033d7dca7eef377f103b1232"
Sep 10 17:34:32.171008 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:32.170948    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71f2c1ce82811e7d40f21d15f13b028c6b443af7033d7dca7eef377f103b1232"} err="failed to get container status \"71f2c1ce82811e7d40f21d15f13b028c6b443af7033d7dca7eef377f103b1232\": rpc error: code = NotFound desc = could not find container \"71f2c1ce82811e7d40f21d15f13b028c6b443af7033d7dca7eef377f103b1232\": container with ID starting with 71f2c1ce82811e7d40f21d15f13b028c6b443af7033d7dca7eef377f103b1232 not found: ID does not exist"
Sep 10 17:34:32.171263 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:34:32.171242    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7914-af378e\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" podUID="3f951752-d2a7-424c-a32d-b356503cd66d" pod="opendatahub/test-curl-7914-af378e"
Sep 10 17:34:32.243687 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:32.243649    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l296d\" (UniqueName: \"kubernetes.io/projected/e721e31b-3d72-47a5-af8d-46fe9ab792d8-kube-api-access-l296d\") pod \"test-curl-7914-615e3a\" (UID: \"e721e31b-3d72-47a5-af8d-46fe9ab792d8\") " pod="opendatahub/test-curl-7914-615e3a"
Sep 10 17:34:32.254115 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:32.254089    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l296d\" (UniqueName: \"kubernetes.io/projected/e721e31b-3d72-47a5-af8d-46fe9ab792d8-kube-api-access-l296d\") pod \"test-curl-7914-615e3a\" (UID: \"e721e31b-3d72-47a5-af8d-46fe9ab792d8\") " pod="opendatahub/test-curl-7914-615e3a"
Sep 10 17:34:32.310231 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:32.310197    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7914-615e3a"
Sep 10 17:34:32.588850 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:32.588819    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7914-615e3a"]
Sep 10 17:34:32.590188 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:34:32.590163    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode721e31b_3d72_47a5_af8d_46fe9ab792d8.slice/crio-5c3c6594357281c210328deda1ad3cb962820c46bdff0d37b2c6b60535a20c07 WatchSource:0}: Error finding container 5c3c6594357281c210328deda1ad3cb962820c46bdff0d37b2c6b60535a20c07: Status 404 returned error can't find the container with id 5c3c6594357281c210328deda1ad3cb962820c46bdff0d37b2c6b60535a20c07
Sep 10 17:34:32.651362 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:32.651258    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:32.834485 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:32.834444    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f951752-d2a7-424c-a32d-b356503cd66d" path="/var/lib/kubelet/pods/3f951752-d2a7-424c-a32d-b356503cd66d/volumes"
Sep 10 17:34:33.080940 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:33.080897    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:33.169477 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:33.169439    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7914-615e3a" event={"ID":"e721e31b-3d72-47a5-af8d-46fe9ab792d8","Type":"ContainerStarted","Data":"b889ce7086775228ded4387e2fcf386208fc9f9696e43b1406d97d1cc00e6701"}
Sep 10 17:34:33.169853 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:33.169491    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7914-615e3a" event={"ID":"e721e31b-3d72-47a5-af8d-46fe9ab792d8","Type":"ContainerStarted","Data":"5c3c6594357281c210328deda1ad3cb962820c46bdff0d37b2c6b60535a20c07"}
Sep 10 17:34:33.170689 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:33.170663    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7914-04ee92" podUID="a4110962-fff5-4e72-9739-bdc221196c30" containerName="test-curl-7914-04ee92" containerID="cri-o://302f3d390d9ad3c52a2de1b05fe78ecccf0b45df5a9e03b834c4469343d362ec" gracePeriod=2
Sep 10 17:34:33.173667 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:33.173641    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:33.188609 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:33.188571    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7914-615e3a" podStartSLOduration=2.188548797 podStartE2EDuration="2.188548797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:34:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:34:33.187889994 +0000 UTC m=+1276.869062939" watchObservedRunningTime="2026-09-10 17:34:33.188548797 +0000 UTC m=+1276.869721767"
Sep 10 17:34:33.325059 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:33.325034    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7914-09d9d3"
Sep 10 17:34:33.456414 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:33.456384    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5fmb\" (UniqueName: \"kubernetes.io/projected/f64beed9-c22c-446e-8134-4a93b0641181-kube-api-access-x5fmb\") pod \"f64beed9-c22c-446e-8134-4a93b0641181\" (UID: \"f64beed9-c22c-446e-8134-4a93b0641181\") "
Sep 10 17:34:33.458242 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:33.458217    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f64beed9-c22c-446e-8134-4a93b0641181-kube-api-access-x5fmb" (OuterVolumeSpecName: "kube-api-access-x5fmb") pod "f64beed9-c22c-446e-8134-4a93b0641181" (UID: "f64beed9-c22c-446e-8134-4a93b0641181"). InnerVolumeSpecName "kube-api-access-x5fmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:34:33.558028 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:33.557998    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5fmb\" (UniqueName: \"kubernetes.io/projected/f64beed9-c22c-446e-8134-4a93b0641181-kube-api-access-x5fmb\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:34:33.808793 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:33.808752    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:33.890735 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:33.890699    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7914-615e3a"]
Sep 10 17:34:33.893755 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:33.893729    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7914-615e3a"]
Sep 10 17:34:34.177754 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:34.177717    2566 generic.go:356] "Generic (PLEG): container finished" podID="f64beed9-c22c-446e-8134-4a93b0641181" containerID="09558590ef78aa705d4ff67247ce67f897d524f91fa02c9126044f2613688f63" exitCode=137
Sep 10 17:34:34.178207 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:34.177778    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7914-09d9d3"
Sep 10 17:34:34.178207 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:34.177831    2566 scope.go:117] "RemoveContainer" containerID="09558590ef78aa705d4ff67247ce67f897d524f91fa02c9126044f2613688f63"
Sep 10 17:34:34.178300 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:34:34.178226    2566 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7914-615e3a" container="test-curl-7914-615e3a"
Sep 10 17:34:34.188186 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:34.188167    2566 scope.go:117] "RemoveContainer" containerID="09558590ef78aa705d4ff67247ce67f897d524f91fa02c9126044f2613688f63"
Sep 10 17:34:34.188471 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:34:34.188446    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7914-09d9d3\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" podUID="f64beed9-c22c-446e-8134-4a93b0641181" pod="opendatahub/test-curl-7914-09d9d3"
Sep 10 17:34:34.188544 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:34:34.188464    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09558590ef78aa705d4ff67247ce67f897d524f91fa02c9126044f2613688f63\": container with ID starting with 09558590ef78aa705d4ff67247ce67f897d524f91fa02c9126044f2613688f63 not found: ID does not exist" containerID="09558590ef78aa705d4ff67247ce67f897d524f91fa02c9126044f2613688f63"
Sep 10 17:34:34.188544 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:34.188493    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09558590ef78aa705d4ff67247ce67f897d524f91fa02c9126044f2613688f63"} err="failed to get container status \"09558590ef78aa705d4ff67247ce67f897d524f91fa02c9126044f2613688f63\": rpc error: code = NotFound desc = could not find container \"09558590ef78aa705d4ff67247ce67f897d524f91fa02c9126044f2613688f63\": container with ID starting with 09558590ef78aa705d4ff67247ce67f897d524f91fa02c9126044f2613688f63 not found: ID does not exist"
Sep 10 17:34:34.498778 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:34.498748    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:34.716582 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:34.716531    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:34.836850 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:34.836765    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f64beed9-c22c-446e-8134-4a93b0641181" path="/var/lib/kubelet/pods/f64beed9-c22c-446e-8134-4a93b0641181/volumes"
Sep 10 17:34:34.904822 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:34.904786    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:35.097183 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:35.097086    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:35.187597 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:35.187569    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7914-615e3a" podUID="e721e31b-3d72-47a5-af8d-46fe9ab792d8" containerName="test-curl-7914-615e3a" containerID="cri-o://b889ce7086775228ded4387e2fcf386208fc9f9696e43b1406d97d1cc00e6701" gracePeriod=2
Sep 10 17:34:35.320750 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:35.320720    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7914-04ee92"
Sep 10 17:34:35.374660 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:35.374583    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ds9br\" (UniqueName: \"kubernetes.io/projected/a4110962-fff5-4e72-9739-bdc221196c30-kube-api-access-ds9br\") pod \"a4110962-fff5-4e72-9739-bdc221196c30\" (UID: \"a4110962-fff5-4e72-9739-bdc221196c30\") "
Sep 10 17:34:35.376697 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:35.376672    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4110962-fff5-4e72-9739-bdc221196c30-kube-api-access-ds9br" (OuterVolumeSpecName: "kube-api-access-ds9br") pod "a4110962-fff5-4e72-9739-bdc221196c30" (UID: "a4110962-fff5-4e72-9739-bdc221196c30"). InnerVolumeSpecName "kube-api-access-ds9br". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:34:35.475814 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:35.475779    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ds9br\" (UniqueName: \"kubernetes.io/projected/a4110962-fff5-4e72-9739-bdc221196c30-kube-api-access-ds9br\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:34:36.105047 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:36.105015    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:36.194747 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:36.194716    2566 generic.go:356] "Generic (PLEG): container finished" podID="a4110962-fff5-4e72-9739-bdc221196c30" containerID="302f3d390d9ad3c52a2de1b05fe78ecccf0b45df5a9e03b834c4469343d362ec" exitCode=137
Sep 10 17:34:36.195192 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:36.194759    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7914-04ee92"
Sep 10 17:34:36.195192 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:36.194813    2566 scope.go:117] "RemoveContainer" containerID="302f3d390d9ad3c52a2de1b05fe78ecccf0b45df5a9e03b834c4469343d362ec"
Sep 10 17:34:36.204365 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:36.204350    2566 scope.go:117] "RemoveContainer" containerID="302f3d390d9ad3c52a2de1b05fe78ecccf0b45df5a9e03b834c4469343d362ec"
Sep 10 17:34:36.204615 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:34:36.204596    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"302f3d390d9ad3c52a2de1b05fe78ecccf0b45df5a9e03b834c4469343d362ec\": container with ID starting with 302f3d390d9ad3c52a2de1b05fe78ecccf0b45df5a9e03b834c4469343d362ec not found: ID does not exist" containerID="302f3d390d9ad3c52a2de1b05fe78ecccf0b45df5a9e03b834c4469343d362ec"
Sep 10 17:34:36.204678 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:36.204625    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"302f3d390d9ad3c52a2de1b05fe78ecccf0b45df5a9e03b834c4469343d362ec"} err="failed to get container status \"302f3d390d9ad3c52a2de1b05fe78ecccf0b45df5a9e03b834c4469343d362ec\": rpc error: code = NotFound desc = could not find container \"302f3d390d9ad3c52a2de1b05fe78ecccf0b45df5a9e03b834c4469343d362ec\": container with ID starting with 302f3d390d9ad3c52a2de1b05fe78ecccf0b45df5a9e03b834c4469343d362ec not found: ID does not exist"
Sep 10 17:34:36.205227 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:34:36.205208    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7914-04ee92\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" podUID="a4110962-fff5-4e72-9739-bdc221196c30" pod="opendatahub/test-curl-7914-04ee92"
Sep 10 17:34:36.834096 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:34:36.834061    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7914-04ee92\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" podUID="a4110962-fff5-4e72-9739-bdc221196c30" pod="opendatahub/test-curl-7914-04ee92"
Sep 10 17:34:36.835198 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:36.835177    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4110962-fff5-4e72-9739-bdc221196c30" path="/var/lib/kubelet/pods/a4110962-fff5-4e72-9739-bdc221196c30/volumes"
Sep 10 17:34:36.977949 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:36.977912    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"]
Sep 10 17:34:36.978849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:36.978822    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e721e31b-3d72-47a5-af8d-46fe9ab792d8" containerName="test-curl-7914-615e3a"
Sep 10 17:34:36.978849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:36.978849    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e721e31b-3d72-47a5-af8d-46fe9ab792d8" containerName="test-curl-7914-615e3a"
Sep 10 17:34:36.979001 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:36.978938    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e721e31b-3d72-47a5-af8d-46fe9ab792d8" containerName="test-curl-7914-615e3a"
Sep 10 17:34:36.983731 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:36.983711    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:34:36.986563 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:36.986538    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-c1fb61-serving-cert\""
Sep 10 17:34:36.986657 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:36.986628    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb27v\" (UniqueName: \"kubernetes.io/projected/9b387931-990a-4093-b866-c8bc2d4d0f8b-kube-api-access-bb27v\") pod \"maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz\" (UID: \"9b387931-990a-4093-b866-c8bc2d4d0f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:34:36.986698 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:36.986674    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b387931-990a-4093-b866-c8bc2d4d0f8b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz\" (UID: \"9b387931-990a-4093-b866-c8bc2d4d0f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:34:36.986761 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:36.986748    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b387931-990a-4093-b866-c8bc2d4d0f8b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz\" (UID: \"9b387931-990a-4093-b866-c8bc2d4d0f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:34:36.986799 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:36.986784    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b387931-990a-4093-b866-c8bc2d4d0f8b-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz\" (UID: \"9b387931-990a-4093-b866-c8bc2d4d0f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:34:36.999934 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:36.999911    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"]
Sep 10 17:34:37.021409 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.021367    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:37.087893 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.087791    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b387931-990a-4093-b866-c8bc2d4d0f8b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz\" (UID: \"9b387931-990a-4093-b866-c8bc2d4d0f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:34:37.087893 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.087842    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b387931-990a-4093-b866-c8bc2d4d0f8b-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz\" (UID: \"9b387931-990a-4093-b866-c8bc2d4d0f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:34:37.088184 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.087907    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb27v\" (UniqueName: \"kubernetes.io/projected/9b387931-990a-4093-b866-c8bc2d4d0f8b-kube-api-access-bb27v\") pod \"maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz\" (UID: \"9b387931-990a-4093-b866-c8bc2d4d0f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:34:37.088184 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.087928    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b387931-990a-4093-b866-c8bc2d4d0f8b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz\" (UID: \"9b387931-990a-4093-b866-c8bc2d4d0f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:34:37.088184 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:34:37.087944    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-serving-cert: secret "maas-api-e2e-shared-b-w3-c1fb61-serving-cert" not found
Sep 10 17:34:37.088184 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:34:37.088018    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b387931-990a-4093-b866-c8bc2d4d0f8b-maas-api-tls podName:9b387931-990a-4093-b866-c8bc2d4d0f8b nodeName:}" failed. No retries permitted until 2026-09-10 17:34:37.587995338 +0000 UTC m=+1281.269168278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9b387931-990a-4093-b866-c8bc2d4d0f8b-maas-api-tls") pod "maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz" (UID: "9b387931-990a-4093-b866-c8bc2d4d0f8b") : secret "maas-api-e2e-shared-b-w3-c1fb61-serving-cert" not found
Sep 10 17:34:37.088664 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.088646    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b387931-990a-4093-b866-c8bc2d4d0f8b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz\" (UID: \"9b387931-990a-4093-b866-c8bc2d4d0f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:34:37.090595 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.090569    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b387931-990a-4093-b866-c8bc2d4d0f8b-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz\" (UID: \"9b387931-990a-4093-b866-c8bc2d4d0f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:34:37.100405 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.100383    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb27v\" (UniqueName: \"kubernetes.io/projected/9b387931-990a-4093-b866-c8bc2d4d0f8b-kube-api-access-bb27v\") pod \"maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz\" (UID: \"9b387931-990a-4093-b866-c8bc2d4d0f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:34:37.111897 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.111871    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:37.223064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.223037    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:37.373167 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.373145    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7914-615e3a"
Sep 10 17:34:37.390372 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.390350    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l296d\" (UniqueName: \"kubernetes.io/projected/e721e31b-3d72-47a5-af8d-46fe9ab792d8-kube-api-access-l296d\") pod \"e721e31b-3d72-47a5-af8d-46fe9ab792d8\" (UID: \"e721e31b-3d72-47a5-af8d-46fe9ab792d8\") "
Sep 10 17:34:37.392391 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.392370    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e721e31b-3d72-47a5-af8d-46fe9ab792d8-kube-api-access-l296d" (OuterVolumeSpecName: "kube-api-access-l296d") pod "e721e31b-3d72-47a5-af8d-46fe9ab792d8" (UID: "e721e31b-3d72-47a5-af8d-46fe9ab792d8"). InnerVolumeSpecName "kube-api-access-l296d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:34:37.491822 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.491771    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l296d\" (UniqueName: \"kubernetes.io/projected/e721e31b-3d72-47a5-af8d-46fe9ab792d8-kube-api-access-l296d\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:34:37.592264 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.592226    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b387931-990a-4093-b866-c8bc2d4d0f8b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz\" (UID: \"9b387931-990a-4093-b866-c8bc2d4d0f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:34:37.594655 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.594634    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b387931-990a-4093-b866-c8bc2d4d0f8b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz\" (UID: \"9b387931-990a-4093-b866-c8bc2d4d0f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:34:37.736839 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.736742    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:37.854776 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.854742    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:37.894264 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:37.894239    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:34:38.179061 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:38.179037    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"]
Sep 10 17:34:38.181237 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:34:38.181207    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b387931_990a_4093_b866_c8bc2d4d0f8b.slice/crio-a07c3f9989b566621f1b4534523902bf4f1d355da6d5628157266198347181ec WatchSource:0}: Error finding container a07c3f9989b566621f1b4534523902bf4f1d355da6d5628157266198347181ec: Status 404 returned error can't find the container with id a07c3f9989b566621f1b4534523902bf4f1d355da6d5628157266198347181ec
Sep 10 17:34:38.208446 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:38.208421    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz" event={"ID":"9b387931-990a-4093-b866-c8bc2d4d0f8b","Type":"ContainerStarted","Data":"a07c3f9989b566621f1b4534523902bf4f1d355da6d5628157266198347181ec"}
Sep 10 17:34:38.209712 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:38.209686    2566 generic.go:356] "Generic (PLEG): container finished" podID="e721e31b-3d72-47a5-af8d-46fe9ab792d8" containerID="b889ce7086775228ded4387e2fcf386208fc9f9696e43b1406d97d1cc00e6701" exitCode=137
Sep 10 17:34:38.209822 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:38.209729    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7914-615e3a"
Sep 10 17:34:38.209822 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:38.209794    2566 scope.go:117] "RemoveContainer" containerID="b889ce7086775228ded4387e2fcf386208fc9f9696e43b1406d97d1cc00e6701"
Sep 10 17:34:38.219533 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:38.219513    2566 scope.go:117] "RemoveContainer" containerID="b889ce7086775228ded4387e2fcf386208fc9f9696e43b1406d97d1cc00e6701"
Sep 10 17:34:38.219806 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:34:38.219783    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b889ce7086775228ded4387e2fcf386208fc9f9696e43b1406d97d1cc00e6701\": container with ID starting with b889ce7086775228ded4387e2fcf386208fc9f9696e43b1406d97d1cc00e6701 not found: ID does not exist" containerID="b889ce7086775228ded4387e2fcf386208fc9f9696e43b1406d97d1cc00e6701"
Sep 10 17:34:38.219875 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:38.219811    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b889ce7086775228ded4387e2fcf386208fc9f9696e43b1406d97d1cc00e6701"} err="failed to get container status \"b889ce7086775228ded4387e2fcf386208fc9f9696e43b1406d97d1cc00e6701\": rpc error: code = NotFound desc = could not find container \"b889ce7086775228ded4387e2fcf386208fc9f9696e43b1406d97d1cc00e6701\": container with ID starting with b889ce7086775228ded4387e2fcf386208fc9f9696e43b1406d97d1cc00e6701 not found: ID does not exist"
Sep 10 17:34:38.220535 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:34:38.220514    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7914-615e3a\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" podUID="e721e31b-3d72-47a5-af8d-46fe9ab792d8" pod="opendatahub/test-curl-7914-615e3a"
Sep 10 17:34:38.306303 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:38.306273    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:38.458924 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:38.458832    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:38.602255 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:38.602227    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:38.835676 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:38.835644    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e721e31b-3d72-47a5-af8d-46fe9ab792d8" path="/var/lib/kubelet/pods/e721e31b-3d72-47a5-af8d-46fe9ab792d8/volumes"
Sep 10 17:34:39.217097 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:39.217014    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz" event={"ID":"9b387931-990a-4093-b866-c8bc2d4d0f8b","Type":"ContainerStarted","Data":"8136f4b40c957859cc46acf651f00278059ece17d29dc4df0006f6da0b04b4ee"}
Sep 10 17:34:39.217097 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:39.217089    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:34:39.250462 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:39.250408    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz" podStartSLOduration=2.852960278 podStartE2EDuration="3.25039223s" totalImagesPullingTime="397.431952ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:34:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:34:39.249877441 +0000 UTC m=+1282.931050383" watchObservedRunningTime="2026-09-10 17:34:39.25039223 +0000 UTC m=+1282.931565173"
Sep 10 17:34:39.655195 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:39.655163    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:40.638600 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:40.638560    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:41.601117 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:41.601083    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:42.583646 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:42.583609    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:43.542792 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.542762    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"]
Sep 10 17:34:43.547414 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.547391    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"
Sep 10 17:34:43.550931 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.550897    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-68cae2e9-serving-cert\""
Sep 10 17:34:43.563925 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.563897    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"]
Sep 10 17:34:43.600031 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.600004    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:43.645060 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.645032    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35961629-d90b-411f-bc2e-9216cae5e017-maas-api-tls\") pod \"maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf\" (UID: \"35961629-d90b-411f-bc2e-9216cae5e017\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"
Sep 10 17:34:43.645231 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.645100    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35961629-d90b-411f-bc2e-9216cae5e017-openshift-service-ca\") pod \"maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf\" (UID: \"35961629-d90b-411f-bc2e-9216cae5e017\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"
Sep 10 17:34:43.645231 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.645117    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpw7w\" (UniqueName: \"kubernetes.io/projected/35961629-d90b-411f-bc2e-9216cae5e017-kube-api-access-xpw7w\") pod \"maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf\" (UID: \"35961629-d90b-411f-bc2e-9216cae5e017\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"
Sep 10 17:34:43.645231 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.645171    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35961629-d90b-411f-bc2e-9216cae5e017-metrics-certs\") pod \"maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf\" (UID: \"35961629-d90b-411f-bc2e-9216cae5e017\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"
Sep 10 17:34:43.703325 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.703299    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:43.746306 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.746273    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35961629-d90b-411f-bc2e-9216cae5e017-maas-api-tls\") pod \"maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf\" (UID: \"35961629-d90b-411f-bc2e-9216cae5e017\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"
Sep 10 17:34:43.746500 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.746393    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35961629-d90b-411f-bc2e-9216cae5e017-openshift-service-ca\") pod \"maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf\" (UID: \"35961629-d90b-411f-bc2e-9216cae5e017\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"
Sep 10 17:34:43.746500 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.746424    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpw7w\" (UniqueName: \"kubernetes.io/projected/35961629-d90b-411f-bc2e-9216cae5e017-kube-api-access-xpw7w\") pod \"maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf\" (UID: \"35961629-d90b-411f-bc2e-9216cae5e017\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"
Sep 10 17:34:43.746500 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.746465    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35961629-d90b-411f-bc2e-9216cae5e017-metrics-certs\") pod \"maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf\" (UID: \"35961629-d90b-411f-bc2e-9216cae5e017\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"
Sep 10 17:34:43.747205 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.747176    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35961629-d90b-411f-bc2e-9216cae5e017-openshift-service-ca\") pod \"maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf\" (UID: \"35961629-d90b-411f-bc2e-9216cae5e017\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"
Sep 10 17:34:43.748841 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.748811    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35961629-d90b-411f-bc2e-9216cae5e017-maas-api-tls\") pod \"maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf\" (UID: \"35961629-d90b-411f-bc2e-9216cae5e017\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"
Sep 10 17:34:43.748949 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.748867    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35961629-d90b-411f-bc2e-9216cae5e017-metrics-certs\") pod \"maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf\" (UID: \"35961629-d90b-411f-bc2e-9216cae5e017\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"
Sep 10 17:34:43.759351 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.759315    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpw7w\" (UniqueName: \"kubernetes.io/projected/35961629-d90b-411f-bc2e-9216cae5e017-kube-api-access-xpw7w\") pod \"maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf\" (UID: \"35961629-d90b-411f-bc2e-9216cae5e017\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"
Sep 10 17:34:43.812213 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.812100    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:43.859139 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:43.859091    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"
Sep 10 17:34:44.138849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:44.138820    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"]
Sep 10 17:34:44.140724 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:34:44.140699    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35961629_d90b_411f_bc2e_9216cae5e017.slice/crio-c39723bee3e558f5caf27fb412e62201d6f50f77b70b02d4c085a8b19cc35e89 WatchSource:0}: Error finding container c39723bee3e558f5caf27fb412e62201d6f50f77b70b02d4c085a8b19cc35e89: Status 404 returned error can't find the container with id c39723bee3e558f5caf27fb412e62201d6f50f77b70b02d4c085a8b19cc35e89
Sep 10 17:34:44.156738 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:44.156701    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:44.252098 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:44.252062    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf" event={"ID":"35961629-d90b-411f-bc2e-9216cae5e017","Type":"ContainerStarted","Data":"c39723bee3e558f5caf27fb412e62201d6f50f77b70b02d4c085a8b19cc35e89"}
Sep 10 17:34:44.265848 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:44.265816    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:45.229645 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:45.229614    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:34:45.260428 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:45.260389    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf" event={"ID":"35961629-d90b-411f-bc2e-9216cae5e017","Type":"ContainerStarted","Data":"9cc63d707667a9bdb020bec4126072b982acb084f00a47ee9714937f787c7857"}
Sep 10 17:34:45.260584 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:45.260546    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"
Sep 10 17:34:45.285000 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:45.284943    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf" podStartSLOduration=1.876946123 podStartE2EDuration="2.284923035s" totalImagesPullingTime="407.976912ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:34:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:34:45.282304814 +0000 UTC m=+1288.963477767" watchObservedRunningTime="2026-09-10 17:34:45.284923035 +0000 UTC m=+1288.966095979"
Sep 10 17:34:51.272143 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:51.272092    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"
Sep 10 17:34:58.242949 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:58.242912    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:34:59.800526 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:34:59.800491    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:35:00.952183 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:00.952147    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:35:01.761989 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:01.761950    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:35:02.859895 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:02.859857    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:35:02.989863 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:02.989822    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:35:03.269646 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:03.269614    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:35:03.782201 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:03.782165    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:35:04.770865 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:04.770815    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:35:06.040913 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:06.040881    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:35:07.160292 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:07.160262    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:35:08.047273 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:08.047237    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:35:11.107141 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:11.107102    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:35:33.795788 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:33.795748    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:35:44.522230 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:44.522133    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:35:54.516812 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.516778    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"]
Sep 10 17:35:54.520781 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.520759    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.523870 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.523849    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-c1fb61\"/\"kube-root-ca.crt\""
Sep 10 17:35:54.524532 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.524517    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-c1fb61\"/\"openshift-service-ca.crt\""
Sep 10 17:35:54.524648 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.524620    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-c1fb61\"/\"default-dockercfg-jc8b9\""
Sep 10 17:35:54.524718 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.524624    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-c1fb61\"/\"auth-test-model-c1fb61-kserve-self-signed-certs\""
Sep 10 17:35:54.534807 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.534776    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"]
Sep 10 17:35:54.562652 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.562616    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:35:54.610849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.610807    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-dshm\") pod \"auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") " pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.610849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.610846    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-tmp-dir\") pod \"auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") " pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.611090 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.610877    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-tls-certs\") pod \"auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") " pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.611090 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.610938    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4l2j\" (UniqueName: \"kubernetes.io/projected/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-kube-api-access-q4l2j\") pod \"auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") " pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.611090 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.611002    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-home\") pod \"auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") " pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.611090 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.611024    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-model-cache\") pod \"auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") " pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.711645 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.711612    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-home\") pod \"auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") " pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.711645 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.711643    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-model-cache\") pod \"auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") " pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.711913 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.711690    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-dshm\") pod \"auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") " pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.711913 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.711726    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-tmp-dir\") pod \"auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") " pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.711913 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.711758    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-tls-certs\") pod \"auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") " pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.711913 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.711802    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4l2j\" (UniqueName: \"kubernetes.io/projected/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-kube-api-access-q4l2j\") pod \"auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") " pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.712150 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.712048    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-home\") pod \"auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") " pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.712150 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.712096    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-model-cache\") pod \"auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") " pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.712269 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.712250    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-tmp-dir\") pod \"auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") " pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.713967 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.713946    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-dshm\") pod \"auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") " pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.714250 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.714234    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-tls-certs\") pod \"auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") " pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.721633 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.721607    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4l2j\" (UniqueName: \"kubernetes.io/projected/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-kube-api-access-q4l2j\") pod \"auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") " pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:54.829684 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:54.829586    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:35:55.109183 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:55.109149    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"]
Sep 10 17:35:55.110574 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:35:55.110550    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc47bc929_641e_442d_aa00_7b4ae7b6b8fc.slice/crio-4e1355453e888b217eed51fdff9b43744ec99cb31be3d9b27a300d25bccbfc5c WatchSource:0}: Error finding container 4e1355453e888b217eed51fdff9b43744ec99cb31be3d9b27a300d25bccbfc5c: Status 404 returned error can't find the container with id 4e1355453e888b217eed51fdff9b43744ec99cb31be3d9b27a300d25bccbfc5c
Sep 10 17:35:55.714453 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:55.714408    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4" event={"ID":"c47bc929-641e-442d-aa00-7b4ae7b6b8fc","Type":"ContainerStarted","Data":"415c9d95a629a532d57aeafdfa84ae24cf88b583b8c5eb83152a3e997441d56b"}
Sep 10 17:35:55.714962 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:55.714467    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4" event={"ID":"c47bc929-641e-442d-aa00-7b4ae7b6b8fc","Type":"ContainerStarted","Data":"4e1355453e888b217eed51fdff9b43744ec99cb31be3d9b27a300d25bccbfc5c"}
Sep 10 17:35:55.733932 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:35:55.733885    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4" podStartSLOduration=1.7338703610000001 podStartE2EDuration="1.733870361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:35:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:35:55.732626397 +0000 UTC m=+1359.413799331" watchObservedRunningTime="2026-09-10 17:35:55.733870361 +0000 UTC m=+1359.415043303"
Sep 10 17:36:03.495654 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:03.495615    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:04.480570 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:04.480535    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:04.836207 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:04.836173    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:36:04.836703 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:04.836216    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:36:04.843109 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:04.843090    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:36:05.405805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:05.405770    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:05.787119 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:05.787089    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:36:06.275685 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:06.275655    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:07.081202 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:07.081163    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:07.561345 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:07.561311    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:07.670613 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:07.670574    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:07.859832 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:07.859748    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:08.668910 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:08.668874    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:17.045224 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:17.045191    2566 scope.go:117] "RemoveContainer" containerID="c32a9d13d390168decaec9a7c4c4fb5aea7fe2410ad3c30b737cee9c70bbd48a"
Sep 10 17:36:17.054165 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:17.054143    2566 scope.go:117] "RemoveContainer" containerID="7497b493929485f5741228e80c6c32bea5968c29510af251f4ec1af3ebd49fb7"
Sep 10 17:36:18.805896 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:18.805863    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"]
Sep 10 17:36:18.806285 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:18.806139    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf" podUID="35961629-d90b-411f-bc2e-9216cae5e017" containerName="maas-api" containerID="cri-o://9cc63d707667a9bdb020bec4126072b982acb084f00a47ee9714937f787c7857" gracePeriod=30
Sep 10 17:36:18.841994 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:18.841966    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:18.922923 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:18.922889    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:19.023310 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.023269    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:19.066737 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.066678    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"
Sep 10 17:36:19.110943 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.110899    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:19.150778 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.150741    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35961629-d90b-411f-bc2e-9216cae5e017-maas-api-tls\") pod \"35961629-d90b-411f-bc2e-9216cae5e017\" (UID: \"35961629-d90b-411f-bc2e-9216cae5e017\") "
Sep 10 17:36:19.150961 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.150843    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35961629-d90b-411f-bc2e-9216cae5e017-metrics-certs\") pod \"35961629-d90b-411f-bc2e-9216cae5e017\" (UID: \"35961629-d90b-411f-bc2e-9216cae5e017\") "
Sep 10 17:36:19.150961 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.150891    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35961629-d90b-411f-bc2e-9216cae5e017-openshift-service-ca\") pod \"35961629-d90b-411f-bc2e-9216cae5e017\" (UID: \"35961629-d90b-411f-bc2e-9216cae5e017\") "
Sep 10 17:36:19.150961 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.150951    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpw7w\" (UniqueName: \"kubernetes.io/projected/35961629-d90b-411f-bc2e-9216cae5e017-kube-api-access-xpw7w\") pod \"35961629-d90b-411f-bc2e-9216cae5e017\" (UID: \"35961629-d90b-411f-bc2e-9216cae5e017\") "
Sep 10 17:36:19.151376 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.151353    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35961629-d90b-411f-bc2e-9216cae5e017-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "35961629-d90b-411f-bc2e-9216cae5e017" (UID: "35961629-d90b-411f-bc2e-9216cae5e017"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:36:19.152980 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.152954    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35961629-d90b-411f-bc2e-9216cae5e017-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "35961629-d90b-411f-bc2e-9216cae5e017" (UID: "35961629-d90b-411f-bc2e-9216cae5e017"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:36:19.153070 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.152992    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35961629-d90b-411f-bc2e-9216cae5e017-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "35961629-d90b-411f-bc2e-9216cae5e017" (UID: "35961629-d90b-411f-bc2e-9216cae5e017"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:36:19.153070 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.153024    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35961629-d90b-411f-bc2e-9216cae5e017-kube-api-access-xpw7w" (OuterVolumeSpecName: "kube-api-access-xpw7w") pod "35961629-d90b-411f-bc2e-9216cae5e017" (UID: "35961629-d90b-411f-bc2e-9216cae5e017"). InnerVolumeSpecName "kube-api-access-xpw7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:36:19.252739 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.252704    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35961629-d90b-411f-bc2e-9216cae5e017-maas-api-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:36:19.252739 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.252731    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35961629-d90b-411f-bc2e-9216cae5e017-metrics-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:36:19.252739 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.252741    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35961629-d90b-411f-bc2e-9216cae5e017-openshift-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:36:19.252993 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.252752    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpw7w\" (UniqueName: \"kubernetes.io/projected/35961629-d90b-411f-bc2e-9216cae5e017-kube-api-access-xpw7w\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:36:19.529561 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.529524    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:19.874840 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.874742    2566 generic.go:356] "Generic (PLEG): container finished" podID="35961629-d90b-411f-bc2e-9216cae5e017" containerID="9cc63d707667a9bdb020bec4126072b982acb084f00a47ee9714937f787c7857" exitCode=0
Sep 10 17:36:19.874840 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.874807    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"
Sep 10 17:36:19.875360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.874828    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf" event={"ID":"35961629-d90b-411f-bc2e-9216cae5e017","Type":"ContainerDied","Data":"9cc63d707667a9bdb020bec4126072b982acb084f00a47ee9714937f787c7857"}
Sep 10 17:36:19.875360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.874899    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf" event={"ID":"35961629-d90b-411f-bc2e-9216cae5e017","Type":"ContainerDied","Data":"c39723bee3e558f5caf27fb412e62201d6f50f77b70b02d4c085a8b19cc35e89"}
Sep 10 17:36:19.875360 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.874932    2566 scope.go:117] "RemoveContainer" containerID="9cc63d707667a9bdb020bec4126072b982acb084f00a47ee9714937f787c7857"
Sep 10 17:36:19.885756 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.885737    2566 scope.go:117] "RemoveContainer" containerID="9cc63d707667a9bdb020bec4126072b982acb084f00a47ee9714937f787c7857"
Sep 10 17:36:19.886040 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:36:19.886022    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cc63d707667a9bdb020bec4126072b982acb084f00a47ee9714937f787c7857\": container with ID starting with 9cc63d707667a9bdb020bec4126072b982acb084f00a47ee9714937f787c7857 not found: ID does not exist" containerID="9cc63d707667a9bdb020bec4126072b982acb084f00a47ee9714937f787c7857"
Sep 10 17:36:19.886084 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.886050    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cc63d707667a9bdb020bec4126072b982acb084f00a47ee9714937f787c7857"} err="failed to get container status \"9cc63d707667a9bdb020bec4126072b982acb084f00a47ee9714937f787c7857\": rpc error: code = NotFound desc = could not find container \"9cc63d707667a9bdb020bec4126072b982acb084f00a47ee9714937f787c7857\": container with ID starting with 9cc63d707667a9bdb020bec4126072b982acb084f00a47ee9714937f787c7857 not found: ID does not exist"
Sep 10 17:36:19.897097 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.897069    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"]
Sep 10 17:36:19.902200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:19.902176    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-68cae2e9-665888fbc8-54mjf"]
Sep 10 17:36:20.285645 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:20.285610    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:20.835311 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:20.835275    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35961629-d90b-411f-bc2e-9216cae5e017" path="/var/lib/kubelet/pods/35961629-d90b-411f-bc2e-9216cae5e017/volumes"
Sep 10 17:36:21.273102 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:21.273065    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:22.214557 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:22.214516    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:22.313870 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:22.313832    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:22.933428 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:22.933381    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:27.254115 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:36:27.254078    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:03.621999 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:03.621957    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:04.312491 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:04.312452    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:09.982236 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:09.982146    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:10.096298 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:10.096259    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:10.270923 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:10.270887    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:11.562950 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:11.562906    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:11.656507 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:11.656471    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:11.755894 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:11.755861    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:12.405746 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:12.405710    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:12.490339 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:12.490294    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:13.768510 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:13.768471    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:19.836513 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:19.836467    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:19.910264 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:19.910226    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:20.488333 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:20.488293    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:26.849990 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:26.849958    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:26.962231 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:26.962183    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:27.048413 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:27.048377    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:27.150222 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:27.150103    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:37.538426 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:37.538388    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:37.755504 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:37.755471    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:58.670104 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:37:58.670061    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:04.450944 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:04.450903    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:04.516846 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:04.516812    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:04.810625 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:04.810588    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:05.124375 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:05.124243    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:06.159611 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:06.159572    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:07.373159 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:07.373101    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:07.464333 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:07.464294    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:08.427463 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:08.427431    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:08.533598 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:08.533560    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:08.671543 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:08.671506    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:09.459417 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:09.459373    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:10.439952 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:10.439912    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:11.303491 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:11.303451    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:12.065069 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.065036    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"]
Sep 10 17:38:12.065578 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.065563    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35961629-d90b-411f-bc2e-9216cae5e017" containerName="maas-api"
Sep 10 17:38:12.065626 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.065581    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="35961629-d90b-411f-bc2e-9216cae5e017" containerName="maas-api"
Sep 10 17:38:12.065679 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.065667    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="35961629-d90b-411f-bc2e-9216cae5e017" containerName="maas-api"
Sep 10 17:38:12.069039 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.069017    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:38:12.071685 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.071660    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-b5f818-serving-cert\""
Sep 10 17:38:12.085703 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.085676    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"]
Sep 10 17:38:12.101803 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.101759    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:12.135979 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.135929    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-openshift-service-ca\") pod \"maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc\" (UID: \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:38:12.136224 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.135998    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-metrics-certs\") pod \"maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc\" (UID: \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:38:12.136224 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.136161    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-maas-api-tls\") pod \"maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc\" (UID: \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:38:12.136224 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.136189    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht2zq\" (UniqueName: \"kubernetes.io/projected/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-kube-api-access-ht2zq\") pod \"maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc\" (UID: \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:38:12.198918 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.198880    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:12.236913 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.236874    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-openshift-service-ca\") pod \"maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc\" (UID: \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:38:12.236913 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.236919    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-metrics-certs\") pod \"maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc\" (UID: \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:38:12.237209 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.236952    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-maas-api-tls\") pod \"maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc\" (UID: \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:38:12.237209 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.236973    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht2zq\" (UniqueName: \"kubernetes.io/projected/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-kube-api-access-ht2zq\") pod \"maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc\" (UID: \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:38:12.237209 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:38:12.237057    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-serving-cert: secret "maas-api-e2e-api-a-b5f818-serving-cert" not found
Sep 10 17:38:12.237209 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:38:12.237175    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-maas-api-tls podName:93f4a0b8-2bbe-42e3-9807-bdf60114b4cf nodeName:}" failed. No retries permitted until 2026-09-10 17:38:12.737152327 +0000 UTC m=+1496.418325263 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-maas-api-tls") pod "maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc" (UID: "93f4a0b8-2bbe-42e3-9807-bdf60114b4cf") : secret "maas-api-e2e-api-a-b5f818-serving-cert" not found
Sep 10 17:38:12.237705 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.237681    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-openshift-service-ca\") pod \"maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc\" (UID: \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:38:12.239595 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.239569    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-metrics-certs\") pod \"maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc\" (UID: \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:38:12.247917 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.247891    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht2zq\" (UniqueName: \"kubernetes.io/projected/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-kube-api-access-ht2zq\") pod \"maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc\" (UID: \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:38:12.601064 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.601023    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:12.701784 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.701746    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:12.741687 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.741654    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-maas-api-tls\") pod \"maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc\" (UID: \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:38:12.744094 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.744069    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-maas-api-tls\") pod \"maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc\" (UID: \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:38:12.980349 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:12.980245    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:38:13.263012 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:13.262988    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"]
Sep 10 17:38:13.264939 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:38:13.264915    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93f4a0b8_2bbe_42e3_9807_bdf60114b4cf.slice/crio-868b74549074a0aa05d3259b1631a67e5e0da04eacf74027705eb82288411c03 WatchSource:0}: Error finding container 868b74549074a0aa05d3259b1631a67e5e0da04eacf74027705eb82288411c03: Status 404 returned error can't find the container with id 868b74549074a0aa05d3259b1631a67e5e0da04eacf74027705eb82288411c03
Sep 10 17:38:13.266386 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:13.266365    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 17:38:13.594599 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:13.594503    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc" event={"ID":"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf","Type":"ContainerStarted","Data":"868b74549074a0aa05d3259b1631a67e5e0da04eacf74027705eb82288411c03"}
Sep 10 17:38:14.602184 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:14.602148    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc" event={"ID":"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf","Type":"ContainerStarted","Data":"9b6d25b9837043c9f8a4218919fa047a59afc05f31a08252bab7c00b32e51d02"}
Sep 10 17:38:14.602642 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:14.602251    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:38:14.622528 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:14.622472    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc" podStartSLOduration=2.175743045 podStartE2EDuration="2.622454128s" totalImagesPullingTime="446.711083ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:38:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:38:14.621249611 +0000 UTC m=+1498.302422554" watchObservedRunningTime="2026-09-10 17:38:14.622454128 +0000 UTC m=+1498.303627118"
Sep 10 17:38:15.422672 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:15.422634    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7917-2496f9"]
Sep 10 17:38:15.426638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:15.426619    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-2496f9"
Sep 10 17:38:15.429614 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:15.429585    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4s82r\""
Sep 10 17:38:15.434534 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:15.434505    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7917-2496f9"]
Sep 10 17:38:15.571179 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:15.571106    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m75ls\" (UniqueName: \"kubernetes.io/projected/4cab56fc-53a4-4494-b0ec-5ba8cea9d443-kube-api-access-m75ls\") pod \"test-curl-7917-2496f9\" (UID: \"4cab56fc-53a4-4494-b0ec-5ba8cea9d443\") " pod="opendatahub/test-curl-7917-2496f9"
Sep 10 17:38:15.672809 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:15.672729    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m75ls\" (UniqueName: \"kubernetes.io/projected/4cab56fc-53a4-4494-b0ec-5ba8cea9d443-kube-api-access-m75ls\") pod \"test-curl-7917-2496f9\" (UID: \"4cab56fc-53a4-4494-b0ec-5ba8cea9d443\") " pod="opendatahub/test-curl-7917-2496f9"
Sep 10 17:38:15.683191 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:15.683159    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m75ls\" (UniqueName: \"kubernetes.io/projected/4cab56fc-53a4-4494-b0ec-5ba8cea9d443-kube-api-access-m75ls\") pod \"test-curl-7917-2496f9\" (UID: \"4cab56fc-53a4-4494-b0ec-5ba8cea9d443\") " pod="opendatahub/test-curl-7917-2496f9"
Sep 10 17:38:15.739196 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:15.739162    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-2496f9"
Sep 10 17:38:16.013635 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:16.013610    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7917-2496f9"]
Sep 10 17:38:16.015368 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:38:16.015330    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4cab56fc_53a4_4494_b0ec_5ba8cea9d443.slice/crio-eb6aad0feba8d2871d39412030065087e9370f0f59f6d5fec1508d03f04719f8 WatchSource:0}: Error finding container eb6aad0feba8d2871d39412030065087e9370f0f59f6d5fec1508d03f04719f8: Status 404 returned error can't find the container with id eb6aad0feba8d2871d39412030065087e9370f0f59f6d5fec1508d03f04719f8
Sep 10 17:38:16.617333 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:16.617289    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7917-2496f9" event={"ID":"4cab56fc-53a4-4494-b0ec-5ba8cea9d443","Type":"ContainerStarted","Data":"fe044db4a407fb5d7073df6bba80598b00786d59ffdc2f35d27f302cb6c7c32e"}
Sep 10 17:38:16.617333 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:16.617335    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7917-2496f9" event={"ID":"4cab56fc-53a4-4494-b0ec-5ba8cea9d443","Type":"ContainerStarted","Data":"eb6aad0feba8d2871d39412030065087e9370f0f59f6d5fec1508d03f04719f8"}
Sep 10 17:38:16.637938 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:16.637855    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7917-2496f9" podStartSLOduration=1.637834064 podStartE2EDuration="1.637834064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:38:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:38:16.632507705 +0000 UTC m=+1500.313680639" watchObservedRunningTime="2026-09-10 17:38:16.637834064 +0000 UTC m=+1500.319007008"
Sep 10 17:38:17.094493 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:17.094465    2566 scope.go:117] "RemoveContainer" containerID="eef3a15e2c32e701e322c60addfc21f1018928caedb326df9f4f541db5632189"
Sep 10 17:38:17.269062 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:17.269029    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7917-2496f9"]
Sep 10 17:38:17.274072 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:17.274046    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7917-2496f9"]
Sep 10 17:38:17.628495 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:38:17.628462    2566 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7917-2496f9" container="test-curl-7917-2496f9"
Sep 10 17:38:18.637301 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:18.637262    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7917-2496f9" podUID="4cab56fc-53a4-4494-b0ec-5ba8cea9d443" containerName="test-curl-7917-2496f9" containerID="cri-o://fe044db4a407fb5d7073df6bba80598b00786d59ffdc2f35d27f302cb6c7c32e" gracePeriod=2
Sep 10 17:38:18.640921 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:38:18.640888    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7917-2496f9\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" podUID="4cab56fc-53a4-4494-b0ec-5ba8cea9d443" pod="opendatahub/test-curl-7917-2496f9"
Sep 10 17:38:18.658967 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:18.658938    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:19.550685 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:19.550650    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"]
Sep 10 17:38:19.568509 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:19.568457    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:20.615816 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:20.615784    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:38:20.823689 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:20.823663    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-2496f9"
Sep 10 17:38:20.928902 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:20.928812    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m75ls\" (UniqueName: \"kubernetes.io/projected/4cab56fc-53a4-4494-b0ec-5ba8cea9d443-kube-api-access-m75ls\") pod \"4cab56fc-53a4-4494-b0ec-5ba8cea9d443\" (UID: \"4cab56fc-53a4-4494-b0ec-5ba8cea9d443\") "
Sep 10 17:38:20.931014 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:20.930988    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cab56fc-53a4-4494-b0ec-5ba8cea9d443-kube-api-access-m75ls" (OuterVolumeSpecName: "kube-api-access-m75ls") pod "4cab56fc-53a4-4494-b0ec-5ba8cea9d443" (UID: "4cab56fc-53a4-4494-b0ec-5ba8cea9d443"). InnerVolumeSpecName "kube-api-access-m75ls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:38:21.030595 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:21.030562    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m75ls\" (UniqueName: \"kubernetes.io/projected/4cab56fc-53a4-4494-b0ec-5ba8cea9d443-kube-api-access-m75ls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:38:21.658898 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:21.658865    2566 generic.go:356] "Generic (PLEG): container finished" podID="4cab56fc-53a4-4494-b0ec-5ba8cea9d443" containerID="fe044db4a407fb5d7073df6bba80598b00786d59ffdc2f35d27f302cb6c7c32e" exitCode=137
Sep 10 17:38:21.659313 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:21.658903    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-2496f9"
Sep 10 17:38:21.659313 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:21.658962    2566 scope.go:117] "RemoveContainer" containerID="fe044db4a407fb5d7073df6bba80598b00786d59ffdc2f35d27f302cb6c7c32e"
Sep 10 17:38:21.668967 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:21.668944    2566 scope.go:117] "RemoveContainer" containerID="fe044db4a407fb5d7073df6bba80598b00786d59ffdc2f35d27f302cb6c7c32e"
Sep 10 17:38:21.669267 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:38:21.669248    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe044db4a407fb5d7073df6bba80598b00786d59ffdc2f35d27f302cb6c7c32e\": container with ID starting with fe044db4a407fb5d7073df6bba80598b00786d59ffdc2f35d27f302cb6c7c32e not found: ID does not exist" containerID="fe044db4a407fb5d7073df6bba80598b00786d59ffdc2f35d27f302cb6c7c32e"
Sep 10 17:38:21.669329 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:21.669277    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe044db4a407fb5d7073df6bba80598b00786d59ffdc2f35d27f302cb6c7c32e"} err="failed to get container status \"fe044db4a407fb5d7073df6bba80598b00786d59ffdc2f35d27f302cb6c7c32e\": rpc error: code = NotFound desc = could not find container \"fe044db4a407fb5d7073df6bba80598b00786d59ffdc2f35d27f302cb6c7c32e\": container with ID starting with fe044db4a407fb5d7073df6bba80598b00786d59ffdc2f35d27f302cb6c7c32e not found: ID does not exist"
Sep 10 17:38:22.835027 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:22.834996    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cab56fc-53a4-4494-b0ec-5ba8cea9d443" path="/var/lib/kubelet/pods/4cab56fc-53a4-4494-b0ec-5ba8cea9d443/volumes"
Sep 10 17:38:34.571027 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.570977    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4" podUID="c47bc929-641e-442d-aa00-7b4ae7b6b8fc" containerName="main" containerID="cri-o://415c9d95a629a532d57aeafdfa84ae24cf88b583b8c5eb83152a3e997441d56b" gracePeriod=45
Sep 10 17:38:34.592601 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.592566    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:34.743058 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.742877    2566 generic.go:356] "Generic (PLEG): container finished" podID="c47bc929-641e-442d-aa00-7b4ae7b6b8fc" containerID="415c9d95a629a532d57aeafdfa84ae24cf88b583b8c5eb83152a3e997441d56b" exitCode=0
Sep 10 17:38:34.743058 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.743010    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4" event={"ID":"c47bc929-641e-442d-aa00-7b4ae7b6b8fc","Type":"ContainerDied","Data":"415c9d95a629a532d57aeafdfa84ae24cf88b583b8c5eb83152a3e997441d56b"}
Sep 10 17:38:34.831423 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.831355    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:38:34.860519 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.860484    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4l2j\" (UniqueName: \"kubernetes.io/projected/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-kube-api-access-q4l2j\") pod \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") "
Sep 10 17:38:34.860729 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.860528    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-home\") pod \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") "
Sep 10 17:38:34.860729 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.860571    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-dshm\") pod \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") "
Sep 10 17:38:34.860729 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.860678    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-tls-certs\") pod \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") "
Sep 10 17:38:34.860729 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.860727    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-tmp-dir\") pod \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") "
Sep 10 17:38:34.860958 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.860761    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-model-cache\") pod \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\" (UID: \"c47bc929-641e-442d-aa00-7b4ae7b6b8fc\") "
Sep 10 17:38:34.861324 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.861283    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c47bc929-641e-442d-aa00-7b4ae7b6b8fc" (UID: "c47bc929-641e-442d-aa00-7b4ae7b6b8fc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:38:34.861324 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.861309    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-model-cache" (OuterVolumeSpecName: "model-cache") pod "c47bc929-641e-442d-aa00-7b4ae7b6b8fc" (UID: "c47bc929-641e-442d-aa00-7b4ae7b6b8fc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:38:34.861520 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.861463    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-home" (OuterVolumeSpecName: "home") pod "c47bc929-641e-442d-aa00-7b4ae7b6b8fc" (UID: "c47bc929-641e-442d-aa00-7b4ae7b6b8fc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:38:34.863394 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.863364    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c47bc929-641e-442d-aa00-7b4ae7b6b8fc" (UID: "c47bc929-641e-442d-aa00-7b4ae7b6b8fc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:34.863394 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.863370    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-dshm" (OuterVolumeSpecName: "dshm") pod "c47bc929-641e-442d-aa00-7b4ae7b6b8fc" (UID: "c47bc929-641e-442d-aa00-7b4ae7b6b8fc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:38:34.863556 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.863457    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-kube-api-access-q4l2j" (OuterVolumeSpecName: "kube-api-access-q4l2j") pod "c47bc929-641e-442d-aa00-7b4ae7b6b8fc" (UID: "c47bc929-641e-442d-aa00-7b4ae7b6b8fc"). InnerVolumeSpecName "kube-api-access-q4l2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:38:34.962022 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.961980    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-tls-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:38:34.962022 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.962013    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-tmp-dir\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:38:34.962022 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.962022    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-model-cache\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:38:34.962022 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.962030    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4l2j\" (UniqueName: \"kubernetes.io/projected/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-kube-api-access-q4l2j\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:38:34.962336 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.962038    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-home\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:38:34.962336 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:34.962048    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c47bc929-641e-442d-aa00-7b4ae7b6b8fc-dshm\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:38:35.526087 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:35.526053    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:35.751032 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:35.750992    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4" event={"ID":"c47bc929-641e-442d-aa00-7b4ae7b6b8fc","Type":"ContainerDied","Data":"4e1355453e888b217eed51fdff9b43744ec99cb31be3d9b27a300d25bccbfc5c"}
Sep 10 17:38:35.751032 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:35.750999    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"
Sep 10 17:38:35.751491 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:35.751039    2566 scope.go:117] "RemoveContainer" containerID="415c9d95a629a532d57aeafdfa84ae24cf88b583b8c5eb83152a3e997441d56b"
Sep 10 17:38:35.773622 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:35.773591    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"]
Sep 10 17:38:35.777592 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:35.777531    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c1fb61/auth-test-model-c1fb61-kserve-7f898bcdcf-hxss4"]
Sep 10 17:38:36.410948 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:36.410908    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:36.841796 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:36.841759    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c47bc929-641e-442d-aa00-7b4ae7b6b8fc" path="/var/lib/kubelet/pods/c47bc929-641e-442d-aa00-7b4ae7b6b8fc/volumes"
Sep 10 17:38:37.186179 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:37.186061    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:38.068310 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:38.068278    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:38.719939 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:38.719903    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:38.809104 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:38.809018    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:38.976341 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:38.976257    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:39.199629 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:39.199591    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:39.551262 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:39.551225    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:39.736138 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:39.736084    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:41.958272 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:41.958215    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:42.671491 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:42.671455    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:42.764241 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:42.764200    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:43.653925 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:43.653877    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:44.579788 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:44.579743    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:45.546805 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:45.546769    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:46.281835 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:46.281801    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:46.982745 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:46.982706    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"]
Sep 10 17:38:46.983375 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:46.983354    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c47bc929-641e-442d-aa00-7b4ae7b6b8fc" containerName="main"
Sep 10 17:38:46.983375 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:46.983372    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c47bc929-641e-442d-aa00-7b4ae7b6b8fc" containerName="main"
Sep 10 17:38:46.983375 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:46.983382    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cab56fc-53a4-4494-b0ec-5ba8cea9d443" containerName="test-curl-7917-2496f9"
Sep 10 17:38:46.983651 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:46.983388    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cab56fc-53a4-4494-b0ec-5ba8cea9d443" containerName="test-curl-7917-2496f9"
Sep 10 17:38:46.983651 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:46.983470    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c47bc929-641e-442d-aa00-7b4ae7b6b8fc" containerName="main"
Sep 10 17:38:46.983651 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:46.983480    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cab56fc-53a4-4494-b0ec-5ba8cea9d443" containerName="test-curl-7917-2496f9"
Sep 10 17:38:46.988491 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:46.988464    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:38:46.991392 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:46.991369    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-a45272-serving-cert\""
Sep 10 17:38:46.998318 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:46.998295    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"]
Sep 10 17:38:47.020149 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.020097    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:47.084729 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.084696    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvdcb\" (UniqueName: \"kubernetes.io/projected/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-kube-api-access-bvdcb\") pod \"maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz\" (UID: \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:38:47.084940 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.084759    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-maas-api-tls\") pod \"maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz\" (UID: \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:38:47.084940 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.084791    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-openshift-service-ca\") pod \"maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz\" (UID: \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:38:47.084940 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.084911    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-metrics-certs\") pod \"maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz\" (UID: \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:38:47.128256 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.128217    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:47.186472 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.186434    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-metrics-certs\") pod \"maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz\" (UID: \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:38:47.186669 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.186489    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvdcb\" (UniqueName: \"kubernetes.io/projected/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-kube-api-access-bvdcb\") pod \"maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz\" (UID: \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:38:47.186669 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.186525    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-maas-api-tls\") pod \"maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz\" (UID: \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:38:47.186669 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.186545    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-openshift-service-ca\") pod \"maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz\" (UID: \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:38:47.186848 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:38:47.186667    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-serving-cert: secret "maas-api-e2e-api-b-a45272-serving-cert" not found
Sep 10 17:38:47.186848 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:38:47.186745    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-maas-api-tls podName:b7bcd5e9-5851-4f19-b62f-d49b86a25a5e nodeName:}" failed. No retries permitted until 2026-09-10 17:38:47.686724864 +0000 UTC m=+1531.367897795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-maas-api-tls") pod "maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz" (UID: "b7bcd5e9-5851-4f19-b62f-d49b86a25a5e") : secret "maas-api-e2e-api-b-a45272-serving-cert" not found
Sep 10 17:38:47.187303 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.187284    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-openshift-service-ca\") pod \"maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz\" (UID: \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:38:47.188935 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.188916    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-metrics-certs\") pod \"maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz\" (UID: \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:38:47.197024 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.196995    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvdcb\" (UniqueName: \"kubernetes.io/projected/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-kube-api-access-bvdcb\") pod \"maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz\" (UID: \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:38:47.512018 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.511968    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:47.628248 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.628216    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:47.692988 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.692948    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-maas-api-tls\") pod \"maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz\" (UID: \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:38:47.695631 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.695604    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-maas-api-tls\") pod \"maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz\" (UID: \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:38:47.724286 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.724257    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:47.904051 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:47.903972    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:38:48.183501 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:48.183450    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"]
Sep 10 17:38:48.185296 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:38:48.185269    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7bcd5e9_5851_4f19_b62f_d49b86a25a5e.slice/crio-6555191c5721a3871b16fce6da648970e89845d680032fca0cf354f28f07d806 WatchSource:0}: Error finding container 6555191c5721a3871b16fce6da648970e89845d680032fca0cf354f28f07d806: Status 404 returned error can't find the container with id 6555191c5721a3871b16fce6da648970e89845d680032fca0cf354f28f07d806
Sep 10 17:38:48.836653 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:48.836565    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz" event={"ID":"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e","Type":"ContainerStarted","Data":"2331110fcdfa651e639301f610669ae5af29b704a5eb8f942bb078b94f9984fc"}
Sep 10 17:38:48.836653 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:48.836631    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz" event={"ID":"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e","Type":"ContainerStarted","Data":"6555191c5721a3871b16fce6da648970e89845d680032fca0cf354f28f07d806"}
Sep 10 17:38:48.836838 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:48.836672    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:38:48.852502 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:48.852459    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz" podStartSLOduration=2.513059329 podStartE2EDuration="2.852444242s" totalImagesPullingTime="339.384913ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:38:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:38:48.849299488 +0000 UTC m=+1532.530472432" watchObservedRunningTime="2026-09-10 17:38:48.852444242 +0000 UTC m=+1532.533617219"
Sep 10 17:38:54.845457 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:38:54.845429    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:39:05.569656 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:05.569615    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:07.034542 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:07.034508    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:08.085328 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:08.085291    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:09.662713 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:09.662672    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:10.756889 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:10.756850    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:11.547621 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:11.547580    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:12.359970 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:12.359929    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:12.680074 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:12.679990    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:13.236620 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:13.236573    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:14.077472 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:14.077435    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:14.913974 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:14.913940    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:16.021735 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:16.021696    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:16.720991 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:16.720952    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:17.120042 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:17.120012    2566 scope.go:117] "RemoveContainer" containerID="c578bc70ca48f7cc9892d760635754b215d3edb629fd9962d99b07663860fec3"
Sep 10 17:39:17.435773 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:17.435651    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:18.215342 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:18.215300    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:18.949848 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:18.949816    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:19.736572 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:19.736533    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:20.466146 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:20.466073    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:21.226070 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:21.226030    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:22.576993 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:22.576952    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:23.681501 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:23.681458    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:25.828478 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:25.828447    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:27.634923 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:27.634892    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:31.458414 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.458380    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"]
Sep 10 17:39:31.458826 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.458605    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc" podUID="93f4a0b8-2bbe-42e3-9807-bdf60114b4cf" containerName="maas-api" containerID="cri-o://9b6d25b9837043c9f8a4218919fa047a59afc05f31a08252bab7c00b32e51d02" gracePeriod=30
Sep 10 17:39:31.492849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.492811    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:31.564145 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.564097    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:31.634646 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.634590    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:31.704062 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.704039    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:39:31.705264 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.705224    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:31.771516 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.771487    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:31.833695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.833671    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-openshift-service-ca\") pod \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\" (UID: \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\") "
Sep 10 17:39:31.833868 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.833711    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-maas-api-tls\") pod \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\" (UID: \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\") "
Sep 10 17:39:31.833868 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.833763    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-metrics-certs\") pod \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\" (UID: \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\") "
Sep 10 17:39:31.833868 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.833793    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ht2zq\" (UniqueName: \"kubernetes.io/projected/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-kube-api-access-ht2zq\") pod \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\" (UID: \"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf\") "
Sep 10 17:39:31.834084 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.834057    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "93f4a0b8-2bbe-42e3-9807-bdf60114b4cf" (UID: "93f4a0b8-2bbe-42e3-9807-bdf60114b4cf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:39:31.835972 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.835937    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-kube-api-access-ht2zq" (OuterVolumeSpecName: "kube-api-access-ht2zq") pod "93f4a0b8-2bbe-42e3-9807-bdf60114b4cf" (UID: "93f4a0b8-2bbe-42e3-9807-bdf60114b4cf"). InnerVolumeSpecName "kube-api-access-ht2zq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:39:31.836189 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.836166    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "93f4a0b8-2bbe-42e3-9807-bdf60114b4cf" (UID: "93f4a0b8-2bbe-42e3-9807-bdf60114b4cf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:39:31.836284 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.836237    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "93f4a0b8-2bbe-42e3-9807-bdf60114b4cf" (UID: "93f4a0b8-2bbe-42e3-9807-bdf60114b4cf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:39:31.934653 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.934621    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-metrics-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:39:31.934906 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.934885    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ht2zq\" (UniqueName: \"kubernetes.io/projected/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-kube-api-access-ht2zq\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:39:31.934987 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.934908    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-openshift-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:39:31.934987 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:31.934923    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf-maas-api-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:39:32.073746 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:32.073649    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:32.102547 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:32.102518    2566 generic.go:356] "Generic (PLEG): container finished" podID="93f4a0b8-2bbe-42e3-9807-bdf60114b4cf" containerID="9b6d25b9837043c9f8a4218919fa047a59afc05f31a08252bab7c00b32e51d02" exitCode=0
Sep 10 17:39:32.102723 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:32.102561    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"
Sep 10 17:39:32.102723 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:32.102596    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc" event={"ID":"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf","Type":"ContainerDied","Data":"9b6d25b9837043c9f8a4218919fa047a59afc05f31a08252bab7c00b32e51d02"}
Sep 10 17:39:32.102723 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:32.102652    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc" event={"ID":"93f4a0b8-2bbe-42e3-9807-bdf60114b4cf","Type":"ContainerDied","Data":"868b74549074a0aa05d3259b1631a67e5e0da04eacf74027705eb82288411c03"}
Sep 10 17:39:32.102723 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:32.102681    2566 scope.go:117] "RemoveContainer" containerID="9b6d25b9837043c9f8a4218919fa047a59afc05f31a08252bab7c00b32e51d02"
Sep 10 17:39:32.112358 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:32.112342    2566 scope.go:117] "RemoveContainer" containerID="9b6d25b9837043c9f8a4218919fa047a59afc05f31a08252bab7c00b32e51d02"
Sep 10 17:39:32.112639 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:39:32.112621    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b6d25b9837043c9f8a4218919fa047a59afc05f31a08252bab7c00b32e51d02\": container with ID starting with 9b6d25b9837043c9f8a4218919fa047a59afc05f31a08252bab7c00b32e51d02 not found: ID does not exist" containerID="9b6d25b9837043c9f8a4218919fa047a59afc05f31a08252bab7c00b32e51d02"
Sep 10 17:39:32.112709 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:32.112653    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b6d25b9837043c9f8a4218919fa047a59afc05f31a08252bab7c00b32e51d02"} err="failed to get container status \"9b6d25b9837043c9f8a4218919fa047a59afc05f31a08252bab7c00b32e51d02\": rpc error: code = NotFound desc = could not find container \"9b6d25b9837043c9f8a4218919fa047a59afc05f31a08252bab7c00b32e51d02\": container with ID starting with 9b6d25b9837043c9f8a4218919fa047a59afc05f31a08252bab7c00b32e51d02 not found: ID does not exist"
Sep 10 17:39:32.125305 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:32.125275    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"]
Sep 10 17:39:32.128529 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:32.128509    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b5f818-78dbdbf6cb-knkdc"]
Sep 10 17:39:32.174529 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:32.174498    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:32.834824 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:32.834789    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93f4a0b8-2bbe-42e3-9807-bdf60114b4cf" path="/var/lib/kubelet/pods/93f4a0b8-2bbe-42e3-9807-bdf60114b4cf/volumes"
Sep 10 17:39:33.060273 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:33.060238    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:33.910787 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:33.910753    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:34.899702 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:34.899666    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:35.806427 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:35.806392    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:36.779723 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:36.779691    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:38.919821 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:38.919791    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:39.018509 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:39.018468    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:39.230982 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:39.230893    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:39.811151 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:39.811097    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:40.785162 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:40.785103    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:41.766328 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:41.766288    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:42.564147 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:42.564070    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:43.362287 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:43.362246    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:44.247664 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:44.247625    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:45.146458 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:45.146423    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:49.193967 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:49.193924    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7917-9a2d8e"]
Sep 10 17:39:49.194473 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:49.194446    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93f4a0b8-2bbe-42e3-9807-bdf60114b4cf" containerName="maas-api"
Sep 10 17:39:49.194473 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:49.194461    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="93f4a0b8-2bbe-42e3-9807-bdf60114b4cf" containerName="maas-api"
Sep 10 17:39:49.194561 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:49.194549    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="93f4a0b8-2bbe-42e3-9807-bdf60114b4cf" containerName="maas-api"
Sep 10 17:39:49.197762 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:49.197743    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-9a2d8e"
Sep 10 17:39:49.200454 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:49.200435    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4s82r\""
Sep 10 17:39:49.205536 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:49.205399    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7917-9a2d8e"]
Sep 10 17:39:49.291901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:49.291857    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdwb8\" (UniqueName: \"kubernetes.io/projected/d0e0380a-6940-418e-8207-7d2bf85c40b4-kube-api-access-jdwb8\") pod \"test-curl-7917-9a2d8e\" (UID: \"d0e0380a-6940-418e-8207-7d2bf85c40b4\") " pod="opendatahub/test-curl-7917-9a2d8e"
Sep 10 17:39:49.393307 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:49.393267    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdwb8\" (UniqueName: \"kubernetes.io/projected/d0e0380a-6940-418e-8207-7d2bf85c40b4-kube-api-access-jdwb8\") pod \"test-curl-7917-9a2d8e\" (UID: \"d0e0380a-6940-418e-8207-7d2bf85c40b4\") " pod="opendatahub/test-curl-7917-9a2d8e"
Sep 10 17:39:49.402486 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:49.402451    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdwb8\" (UniqueName: \"kubernetes.io/projected/d0e0380a-6940-418e-8207-7d2bf85c40b4-kube-api-access-jdwb8\") pod \"test-curl-7917-9a2d8e\" (UID: \"d0e0380a-6940-418e-8207-7d2bf85c40b4\") " pod="opendatahub/test-curl-7917-9a2d8e"
Sep 10 17:39:49.508853 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:49.508812    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-9a2d8e"
Sep 10 17:39:49.784295 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:49.784268    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7917-9a2d8e"]
Sep 10 17:39:49.785872 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:39:49.785849    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0e0380a_6940_418e_8207_7d2bf85c40b4.slice/crio-e7f44fbd5599c53e9b798178e375dc79433225de92de8d6ef2ce61c5a616a444 WatchSource:0}: Error finding container e7f44fbd5599c53e9b798178e375dc79433225de92de8d6ef2ce61c5a616a444: Status 404 returned error can't find the container with id e7f44fbd5599c53e9b798178e375dc79433225de92de8d6ef2ce61c5a616a444
Sep 10 17:39:50.220895 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:50.220790    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7917-9a2d8e" event={"ID":"d0e0380a-6940-418e-8207-7d2bf85c40b4","Type":"ContainerStarted","Data":"8066ca6c1dfc62206db4d145b015091553a91542ab2a018447600b79d18e8287"}
Sep 10 17:39:50.220895 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:50.220844    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7917-9a2d8e" event={"ID":"d0e0380a-6940-418e-8207-7d2bf85c40b4","Type":"ContainerStarted","Data":"e7f44fbd5599c53e9b798178e375dc79433225de92de8d6ef2ce61c5a616a444"}
Sep 10 17:39:50.246444 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:50.246378    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7917-9a2d8e" podStartSLOduration=1.246357634 podStartE2EDuration="1.246357634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:39:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:39:50.243279291 +0000 UTC m=+1593.924452229" watchObservedRunningTime="2026-09-10 17:39:50.246357634 +0000 UTC m=+1593.927530577"
Sep 10 17:39:50.846039 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:50.845972    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7917-9a2d8e"]
Sep 10 17:39:50.849084 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:50.849050    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7917-9a2d8e"]
Sep 10 17:39:50.963990 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:50.963901    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7917-cc267a"]
Sep 10 17:39:50.964442 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:50.964425    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0e0380a-6940-418e-8207-7d2bf85c40b4" containerName="test-curl-7917-9a2d8e"
Sep 10 17:39:50.964495 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:50.964445    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e0380a-6940-418e-8207-7d2bf85c40b4" containerName="test-curl-7917-9a2d8e"
Sep 10 17:39:50.964531 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:50.964514    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0e0380a-6940-418e-8207-7d2bf85c40b4" containerName="test-curl-7917-9a2d8e"
Sep 10 17:39:50.967878 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:50.967861    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-cc267a"
Sep 10 17:39:50.973309 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:50.973284    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7917-cc267a"]
Sep 10 17:39:51.009685 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:51.009641    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ljqn\" (UniqueName: \"kubernetes.io/projected/72bf4cca-3f5a-4014-a7d2-4fbf06c30571-kube-api-access-7ljqn\") pod \"test-curl-7917-cc267a\" (UID: \"72bf4cca-3f5a-4014-a7d2-4fbf06c30571\") " pod="opendatahub/test-curl-7917-cc267a"
Sep 10 17:39:51.111203 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:51.111111    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ljqn\" (UniqueName: \"kubernetes.io/projected/72bf4cca-3f5a-4014-a7d2-4fbf06c30571-kube-api-access-7ljqn\") pod \"test-curl-7917-cc267a\" (UID: \"72bf4cca-3f5a-4014-a7d2-4fbf06c30571\") " pod="opendatahub/test-curl-7917-cc267a"
Sep 10 17:39:51.120920 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:51.120878    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ljqn\" (UniqueName: \"kubernetes.io/projected/72bf4cca-3f5a-4014-a7d2-4fbf06c30571-kube-api-access-7ljqn\") pod \"test-curl-7917-cc267a\" (UID: \"72bf4cca-3f5a-4014-a7d2-4fbf06c30571\") " pod="opendatahub/test-curl-7917-cc267a"
Sep 10 17:39:51.226895 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:39:51.226806    2566 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7917-9a2d8e" container="test-curl-7917-9a2d8e"
Sep 10 17:39:51.279639 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:51.279606    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-cc267a"
Sep 10 17:39:51.555811 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:51.555782    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7917-cc267a"]
Sep 10 17:39:51.557288 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:39:51.557256    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72bf4cca_3f5a_4014_a7d2_4fbf06c30571.slice/crio-2c4f1a45d1acf685817ea4a26a9e0f6df343336176d811a690c6c413f7a7bb92 WatchSource:0}: Error finding container 2c4f1a45d1acf685817ea4a26a9e0f6df343336176d811a690c6c413f7a7bb92: Status 404 returned error can't find the container with id 2c4f1a45d1acf685817ea4a26a9e0f6df343336176d811a690c6c413f7a7bb92
Sep 10 17:39:52.234836 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:52.234794    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7917-cc267a" event={"ID":"72bf4cca-3f5a-4014-a7d2-4fbf06c30571","Type":"ContainerStarted","Data":"19263fbf87c0c63b2aeb795bbed9072393a0dc516cdb55c7629f1c18c1e59d08"}
Sep 10 17:39:52.235343 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:52.234849    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7917-cc267a" event={"ID":"72bf4cca-3f5a-4014-a7d2-4fbf06c30571","Type":"ContainerStarted","Data":"2c4f1a45d1acf685817ea4a26a9e0f6df343336176d811a690c6c413f7a7bb92"}
Sep 10 17:39:52.235343 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:52.234941    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7917-9a2d8e" podUID="d0e0380a-6940-418e-8207-7d2bf85c40b4" containerName="test-curl-7917-9a2d8e" containerID="cri-o://8066ca6c1dfc62206db4d145b015091553a91542ab2a018447600b79d18e8287" gracePeriod=2
Sep 10 17:39:52.252489 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:52.252434    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7917-cc267a" podStartSLOduration=2.252415237 podStartE2EDuration="2.252415237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:39:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:39:52.250501175 +0000 UTC m=+1595.931674138" watchObservedRunningTime="2026-09-10 17:39:52.252415237 +0000 UTC m=+1595.933588180"
Sep 10 17:39:52.252882 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:39:52.252854    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7917-9a2d8e\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" podUID="d0e0380a-6940-418e-8207-7d2bf85c40b4" pod="opendatahub/test-curl-7917-9a2d8e"
Sep 10 17:39:52.254952 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:39:52.254924    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7917-9a2d8e\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" podUID="d0e0380a-6940-418e-8207-7d2bf85c40b4" pod="opendatahub/test-curl-7917-9a2d8e"
Sep 10 17:39:52.851710 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:52.851674    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7917-cc267a"]
Sep 10 17:39:52.856698 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:52.856670    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7917-cc267a"]
Sep 10 17:39:53.241379 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:39:53.241344    2566 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7917-cc267a" container="test-curl-7917-cc267a"
Sep 10 17:39:54.248582 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:54.248546    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7917-cc267a" podUID="72bf4cca-3f5a-4014-a7d2-4fbf06c30571" containerName="test-curl-7917-cc267a" containerID="cri-o://19263fbf87c0c63b2aeb795bbed9072393a0dc516cdb55c7629f1c18c1e59d08" gracePeriod=2
Sep 10 17:39:54.250898 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:39:54.250861    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7917-cc267a\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" podUID="72bf4cca-3f5a-4014-a7d2-4fbf06c30571" pod="opendatahub/test-curl-7917-cc267a"
Sep 10 17:39:54.420807 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:54.420776    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-9a2d8e"
Sep 10 17:39:54.423231 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:39:54.423200    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7917-cc267a\" is forbidden: User \"system:node:ip-10-0-138-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-123.ec2.internal' and this object" podUID="72bf4cca-3f5a-4014-a7d2-4fbf06c30571" pod="opendatahub/test-curl-7917-cc267a"
Sep 10 17:39:54.545826 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:54.545788    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdwb8\" (UniqueName: \"kubernetes.io/projected/d0e0380a-6940-418e-8207-7d2bf85c40b4-kube-api-access-jdwb8\") pod \"d0e0380a-6940-418e-8207-7d2bf85c40b4\" (UID: \"d0e0380a-6940-418e-8207-7d2bf85c40b4\") "
Sep 10 17:39:54.548087 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:54.548049    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0e0380a-6940-418e-8207-7d2bf85c40b4-kube-api-access-jdwb8" (OuterVolumeSpecName: "kube-api-access-jdwb8") pod "d0e0380a-6940-418e-8207-7d2bf85c40b4" (UID: "d0e0380a-6940-418e-8207-7d2bf85c40b4"). InnerVolumeSpecName "kube-api-access-jdwb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:39:54.646712 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:54.646673    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdwb8\" (UniqueName: \"kubernetes.io/projected/d0e0380a-6940-418e-8207-7d2bf85c40b4-kube-api-access-jdwb8\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:39:54.837522 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:54.837421    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0e0380a-6940-418e-8207-7d2bf85c40b4" path="/var/lib/kubelet/pods/d0e0380a-6940-418e-8207-7d2bf85c40b4/volumes"
Sep 10 17:39:54.894002 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:54.893958    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:55.207941 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.207824    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"]
Sep 10 17:39:55.208157 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.208075    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz" podUID="b7bcd5e9-5851-4f19-b62f-d49b86a25a5e" containerName="maas-api" containerID="cri-o://2331110fcdfa651e639301f610669ae5af29b704a5eb8f942bb078b94f9984fc" gracePeriod=30
Sep 10 17:39:55.256160 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.256095    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:55.257327 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.257302    2566 generic.go:356] "Generic (PLEG): container finished" podID="d0e0380a-6940-418e-8207-7d2bf85c40b4" containerID="8066ca6c1dfc62206db4d145b015091553a91542ab2a018447600b79d18e8287" exitCode=137
Sep 10 17:39:55.257452 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.257358    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-9a2d8e"
Sep 10 17:39:55.257452 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.257404    2566 scope.go:117] "RemoveContainer" containerID="8066ca6c1dfc62206db4d145b015091553a91542ab2a018447600b79d18e8287"
Sep 10 17:39:55.325794 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.325766    2566 scope.go:117] "RemoveContainer" containerID="8066ca6c1dfc62206db4d145b015091553a91542ab2a018447600b79d18e8287"
Sep 10 17:39:55.326324 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:39:55.326300    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8066ca6c1dfc62206db4d145b015091553a91542ab2a018447600b79d18e8287\": container with ID starting with 8066ca6c1dfc62206db4d145b015091553a91542ab2a018447600b79d18e8287 not found: ID does not exist" containerID="8066ca6c1dfc62206db4d145b015091553a91542ab2a018447600b79d18e8287"
Sep 10 17:39:55.326420 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.326336    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8066ca6c1dfc62206db4d145b015091553a91542ab2a018447600b79d18e8287"} err="failed to get container status \"8066ca6c1dfc62206db4d145b015091553a91542ab2a018447600b79d18e8287\": rpc error: code = NotFound desc = could not find container \"8066ca6c1dfc62206db4d145b015091553a91542ab2a018447600b79d18e8287\": container with ID starting with 8066ca6c1dfc62206db4d145b015091553a91542ab2a018447600b79d18e8287 not found: ID does not exist"
Sep 10 17:39:55.404389 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.404348    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:55.479190 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.479161    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:39:55.529651 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.529562    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:55.555620 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.555568    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-maas-api-tls\") pod \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\" (UID: \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\") "
Sep 10 17:39:55.555821 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.555721    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-openshift-service-ca\") pod \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\" (UID: \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\") "
Sep 10 17:39:55.555821 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.555779    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvdcb\" (UniqueName: \"kubernetes.io/projected/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-kube-api-access-bvdcb\") pod \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\" (UID: \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\") "
Sep 10 17:39:55.555963 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.555825    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-metrics-certs\") pod \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\" (UID: \"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e\") "
Sep 10 17:39:55.556273 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.556239    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b7bcd5e9-5851-4f19-b62f-d49b86a25a5e" (UID: "b7bcd5e9-5851-4f19-b62f-d49b86a25a5e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:39:55.558052 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.558018    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b7bcd5e9-5851-4f19-b62f-d49b86a25a5e" (UID: "b7bcd5e9-5851-4f19-b62f-d49b86a25a5e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:39:55.558375 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.558348    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b7bcd5e9-5851-4f19-b62f-d49b86a25a5e" (UID: "b7bcd5e9-5851-4f19-b62f-d49b86a25a5e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:39:55.558469 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.558379    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-kube-api-access-bvdcb" (OuterVolumeSpecName: "kube-api-access-bvdcb") pod "b7bcd5e9-5851-4f19-b62f-d49b86a25a5e" (UID: "b7bcd5e9-5851-4f19-b62f-d49b86a25a5e"). InnerVolumeSpecName "kube-api-access-bvdcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:39:55.620145 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.620086    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:55.657319 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.657275    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-openshift-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:39:55.657319 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.657309    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvdcb\" (UniqueName: \"kubernetes.io/projected/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-kube-api-access-bvdcb\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:39:55.657319 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.657326    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-metrics-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:39:55.657562 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.657338    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e-maas-api-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:39:55.705534 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:55.705493    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:56.084812 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:56.084767    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:56.265448 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:56.265417    2566 generic.go:356] "Generic (PLEG): container finished" podID="b7bcd5e9-5851-4f19-b62f-d49b86a25a5e" containerID="2331110fcdfa651e639301f610669ae5af29b704a5eb8f942bb078b94f9984fc" exitCode=0
Sep 10 17:39:56.265851 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:56.265476    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"
Sep 10 17:39:56.265851 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:56.265501    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz" event={"ID":"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e","Type":"ContainerDied","Data":"2331110fcdfa651e639301f610669ae5af29b704a5eb8f942bb078b94f9984fc"}
Sep 10 17:39:56.265851 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:56.265573    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz" event={"ID":"b7bcd5e9-5851-4f19-b62f-d49b86a25a5e","Type":"ContainerDied","Data":"6555191c5721a3871b16fce6da648970e89845d680032fca0cf354f28f07d806"}
Sep 10 17:39:56.265851 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:56.265609    2566 scope.go:117] "RemoveContainer" containerID="2331110fcdfa651e639301f610669ae5af29b704a5eb8f942bb078b94f9984fc"
Sep 10 17:39:56.288627 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:56.288595    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:56.316966 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:56.316937    2566 scope.go:117] "RemoveContainer" containerID="2331110fcdfa651e639301f610669ae5af29b704a5eb8f942bb078b94f9984fc"
Sep 10 17:39:56.317543 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:39:56.317506    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2331110fcdfa651e639301f610669ae5af29b704a5eb8f942bb078b94f9984fc\": container with ID starting with 2331110fcdfa651e639301f610669ae5af29b704a5eb8f942bb078b94f9984fc not found: ID does not exist" containerID="2331110fcdfa651e639301f610669ae5af29b704a5eb8f942bb078b94f9984fc"
Sep 10 17:39:56.317654 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:56.317556    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2331110fcdfa651e639301f610669ae5af29b704a5eb8f942bb078b94f9984fc"} err="failed to get container status \"2331110fcdfa651e639301f610669ae5af29b704a5eb8f942bb078b94f9984fc\": rpc error: code = NotFound desc = could not find container \"2331110fcdfa651e639301f610669ae5af29b704a5eb8f942bb078b94f9984fc\": container with ID starting with 2331110fcdfa651e639301f610669ae5af29b704a5eb8f942bb078b94f9984fc not found: ID does not exist"
Sep 10 17:39:56.336728 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:56.336634    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"]
Sep 10 17:39:56.342248 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:56.342202    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a45272-7575f6dbcd-wc6wz"]
Sep 10 17:39:56.434074 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:56.434048    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-cc267a"
Sep 10 17:39:56.566524 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:56.566487    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ljqn\" (UniqueName: \"kubernetes.io/projected/72bf4cca-3f5a-4014-a7d2-4fbf06c30571-kube-api-access-7ljqn\") pod \"72bf4cca-3f5a-4014-a7d2-4fbf06c30571\" (UID: \"72bf4cca-3f5a-4014-a7d2-4fbf06c30571\") "
Sep 10 17:39:56.568668 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:56.568633    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72bf4cca-3f5a-4014-a7d2-4fbf06c30571-kube-api-access-7ljqn" (OuterVolumeSpecName: "kube-api-access-7ljqn") pod "72bf4cca-3f5a-4014-a7d2-4fbf06c30571" (UID: "72bf4cca-3f5a-4014-a7d2-4fbf06c30571"). InnerVolumeSpecName "kube-api-access-7ljqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:39:56.667458 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:56.667361    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ljqn\" (UniqueName: \"kubernetes.io/projected/72bf4cca-3f5a-4014-a7d2-4fbf06c30571-kube-api-access-7ljqn\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:39:56.835241 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:56.835208    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72bf4cca-3f5a-4014-a7d2-4fbf06c30571" path="/var/lib/kubelet/pods/72bf4cca-3f5a-4014-a7d2-4fbf06c30571/volumes"
Sep 10 17:39:56.835681 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:56.835661    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7bcd5e9-5851-4f19-b62f-d49b86a25a5e" path="/var/lib/kubelet/pods/b7bcd5e9-5851-4f19-b62f-d49b86a25a5e/volumes"
Sep 10 17:39:56.944638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:56.944534    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:57.209667 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:57.209569    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:57.275244 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:57.275209    2566 generic.go:356] "Generic (PLEG): container finished" podID="72bf4cca-3f5a-4014-a7d2-4fbf06c30571" containerID="19263fbf87c0c63b2aeb795bbed9072393a0dc516cdb55c7629f1c18c1e59d08" exitCode=137
Sep 10 17:39:57.275656 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:57.275254    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-cc267a"
Sep 10 17:39:57.275656 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:57.275313    2566 scope.go:117] "RemoveContainer" containerID="19263fbf87c0c63b2aeb795bbed9072393a0dc516cdb55c7629f1c18c1e59d08"
Sep 10 17:39:57.285522 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:57.285457    2566 scope.go:117] "RemoveContainer" containerID="19263fbf87c0c63b2aeb795bbed9072393a0dc516cdb55c7629f1c18c1e59d08"
Sep 10 17:39:57.285833 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:39:57.285805    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19263fbf87c0c63b2aeb795bbed9072393a0dc516cdb55c7629f1c18c1e59d08\": container with ID starting with 19263fbf87c0c63b2aeb795bbed9072393a0dc516cdb55c7629f1c18c1e59d08 not found: ID does not exist" containerID="19263fbf87c0c63b2aeb795bbed9072393a0dc516cdb55c7629f1c18c1e59d08"
Sep 10 17:39:57.285920 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:57.285846    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19263fbf87c0c63b2aeb795bbed9072393a0dc516cdb55c7629f1c18c1e59d08"} err="failed to get container status \"19263fbf87c0c63b2aeb795bbed9072393a0dc516cdb55c7629f1c18c1e59d08\": rpc error: code = NotFound desc = could not find container \"19263fbf87c0c63b2aeb795bbed9072393a0dc516cdb55c7629f1c18c1e59d08\": container with ID starting with 19263fbf87c0c63b2aeb795bbed9072393a0dc516cdb55c7629f1c18c1e59d08 not found: ID does not exist"
Sep 10 17:39:58.151228 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:58.151185    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:59.045733 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:59.045696    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:59.892379 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:39:59.892337    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:00.779072 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:00.779036    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:02.359430 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:02.359396    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:02.459624 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:02.459592    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:02.627849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:02.627761    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:03.085708 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.085668    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:03.121527 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.121484    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"]
Sep 10 17:40:03.122191 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.122172    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72bf4cca-3f5a-4014-a7d2-4fbf06c30571" containerName="test-curl-7917-cc267a"
Sep 10 17:40:03.122238 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.122194    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="72bf4cca-3f5a-4014-a7d2-4fbf06c30571" containerName="test-curl-7917-cc267a"
Sep 10 17:40:03.122238 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.122223    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7bcd5e9-5851-4f19-b62f-d49b86a25a5e" containerName="maas-api"
Sep 10 17:40:03.122238 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.122232    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7bcd5e9-5851-4f19-b62f-d49b86a25a5e" containerName="maas-api"
Sep 10 17:40:03.122357 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.122345    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="72bf4cca-3f5a-4014-a7d2-4fbf06c30571" containerName="test-curl-7917-cc267a"
Sep 10 17:40:03.122397 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.122362    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7bcd5e9-5851-4f19-b62f-d49b86a25a5e" containerName="maas-api"
Sep 10 17:40:03.127155 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.127116    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.130219 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.130193    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-3c30a2\"/\"rate-test-model-3c30a2-kserve-self-signed-certs\""
Sep 10 17:40:03.131244 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.131213    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-3c30a2\"/\"openshift-service-ca.crt\""
Sep 10 17:40:03.131369 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.131271    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-3c30a2\"/\"kube-root-ca.crt\""
Sep 10 17:40:03.131369 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.131272    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-3c30a2\"/\"default-dockercfg-r5g7h\""
Sep 10 17:40:03.139733 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.139706    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"]
Sep 10 17:40:03.181640 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.181597    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:03.240901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.240863    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-dshm\") pod \"rate-test-model-3c30a2-kserve-99474996c-nj7h5\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") " pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.241082 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.240916    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-tmp-dir\") pod \"rate-test-model-3c30a2-kserve-99474996c-nj7h5\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") " pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.241082 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.240976    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-home\") pod \"rate-test-model-3c30a2-kserve-99474996c-nj7h5\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") " pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.241082 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.240995    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-model-cache\") pod \"rate-test-model-3c30a2-kserve-99474996c-nj7h5\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") " pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.241249 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.241089    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7dwf\" (UniqueName: \"kubernetes.io/projected/3505f047-2a6a-48c8-b243-d722453c4d80-kube-api-access-f7dwf\") pod \"rate-test-model-3c30a2-kserve-99474996c-nj7h5\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") " pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.241249 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.241157    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3505f047-2a6a-48c8-b243-d722453c4d80-tls-certs\") pod \"rate-test-model-3c30a2-kserve-99474996c-nj7h5\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") " pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.342292 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.342201    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3505f047-2a6a-48c8-b243-d722453c4d80-tls-certs\") pod \"rate-test-model-3c30a2-kserve-99474996c-nj7h5\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") " pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.342292 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.342264    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-dshm\") pod \"rate-test-model-3c30a2-kserve-99474996c-nj7h5\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") " pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.342505 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.342298    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-tmp-dir\") pod \"rate-test-model-3c30a2-kserve-99474996c-nj7h5\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") " pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.342505 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.342333    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-home\") pod \"rate-test-model-3c30a2-kserve-99474996c-nj7h5\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") " pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.342505 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.342357    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-model-cache\") pod \"rate-test-model-3c30a2-kserve-99474996c-nj7h5\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") " pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.342505 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.342443    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7dwf\" (UniqueName: \"kubernetes.io/projected/3505f047-2a6a-48c8-b243-d722453c4d80-kube-api-access-f7dwf\") pod \"rate-test-model-3c30a2-kserve-99474996c-nj7h5\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") " pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.342780 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.342743    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-tmp-dir\") pod \"rate-test-model-3c30a2-kserve-99474996c-nj7h5\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") " pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.342904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.342815    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-home\") pod \"rate-test-model-3c30a2-kserve-99474996c-nj7h5\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") " pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.342904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.342848    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-model-cache\") pod \"rate-test-model-3c30a2-kserve-99474996c-nj7h5\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") " pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.344708 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.344684    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-dshm\") pod \"rate-test-model-3c30a2-kserve-99474996c-nj7h5\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") " pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.344913 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.344896    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3505f047-2a6a-48c8-b243-d722453c4d80-tls-certs\") pod \"rate-test-model-3c30a2-kserve-99474996c-nj7h5\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") " pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.351993 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.351969    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7dwf\" (UniqueName: \"kubernetes.io/projected/3505f047-2a6a-48c8-b243-d722453c4d80-kube-api-access-f7dwf\") pod \"rate-test-model-3c30a2-kserve-99474996c-nj7h5\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") " pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.437687 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.437652    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:03.737920 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:03.737891    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"]
Sep 10 17:40:03.741158 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:40:03.741099    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3505f047_2a6a_48c8_b243_d722453c4d80.slice/crio-a7068fab434d97ff33269b22618e549f946917fd392c5404d3b72c25b71d37fd WatchSource:0}: Error finding container a7068fab434d97ff33269b22618e549f946917fd392c5404d3b72c25b71d37fd: Status 404 returned error can't find the container with id a7068fab434d97ff33269b22618e549f946917fd392c5404d3b72c25b71d37fd
Sep 10 17:40:04.276278 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:04.276239    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:04.324532 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:04.324487    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5" event={"ID":"3505f047-2a6a-48c8-b243-d722453c4d80","Type":"ContainerStarted","Data":"b5c3ef26153b3ac62af03eb9f530d47396ae9470967e404909d764cd7027d5a0"}
Sep 10 17:40:04.324532 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:04.324539    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5" event={"ID":"3505f047-2a6a-48c8-b243-d722453c4d80","Type":"ContainerStarted","Data":"a7068fab434d97ff33269b22618e549f946917fd392c5404d3b72c25b71d37fd"}
Sep 10 17:40:04.348404 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:04.348344    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5" podStartSLOduration=1.348326238 podStartE2EDuration="1.348326238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:40:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:04.346259513 +0000 UTC m=+1608.027432456" watchObservedRunningTime="2026-09-10 17:40:04.348326238 +0000 UTC m=+1608.029499181"
Sep 10 17:40:05.170656 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:05.170618    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:05.910574 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:05.910523    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:06.725035 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:06.724996    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:07.453904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:07.453866    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:13.438990 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.438721    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:13.440400 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.440367    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:13.458180 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.458141    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"]
Sep 10 17:40:13.458710 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.458651    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq" podUID="1170928a-f7e9-4b2a-bc57-106d01bdaf88" containerName="maas-api" containerID="cri-o://479c6c6e707d5df003e8910fecf6cac479bd26b19a0a5a70a040c804f8fc805e" gracePeriod=30
Sep 10 17:40:13.470450 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.470420    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:13.477368 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.477322    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:13.625962 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.625925    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:13.697364 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.697328    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:13.721017 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.720990    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:40:13.756306 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.756274    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1170928a-f7e9-4b2a-bc57-106d01bdaf88-maas-api-tls\") pod \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\" (UID: \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\") "
Sep 10 17:40:13.756513 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.756327    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1170928a-f7e9-4b2a-bc57-106d01bdaf88-metrics-certs\") pod \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\" (UID: \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\") "
Sep 10 17:40:13.756513 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.756410    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-499r9\" (UniqueName: \"kubernetes.io/projected/1170928a-f7e9-4b2a-bc57-106d01bdaf88-kube-api-access-499r9\") pod \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\" (UID: \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\") "
Sep 10 17:40:13.756513 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.756442    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1170928a-f7e9-4b2a-bc57-106d01bdaf88-openshift-service-ca\") pod \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\" (UID: \"1170928a-f7e9-4b2a-bc57-106d01bdaf88\") "
Sep 10 17:40:13.757011 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.756969    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1170928a-f7e9-4b2a-bc57-106d01bdaf88-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1170928a-f7e9-4b2a-bc57-106d01bdaf88" (UID: "1170928a-f7e9-4b2a-bc57-106d01bdaf88"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:40:13.759049 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.759024    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1170928a-f7e9-4b2a-bc57-106d01bdaf88-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1170928a-f7e9-4b2a-bc57-106d01bdaf88" (UID: "1170928a-f7e9-4b2a-bc57-106d01bdaf88"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:40:13.759174 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.759047    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1170928a-f7e9-4b2a-bc57-106d01bdaf88-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1170928a-f7e9-4b2a-bc57-106d01bdaf88" (UID: "1170928a-f7e9-4b2a-bc57-106d01bdaf88"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:40:13.759216 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.759143    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1170928a-f7e9-4b2a-bc57-106d01bdaf88-kube-api-access-499r9" (OuterVolumeSpecName: "kube-api-access-499r9") pod "1170928a-f7e9-4b2a-bc57-106d01bdaf88" (UID: "1170928a-f7e9-4b2a-bc57-106d01bdaf88"). InnerVolumeSpecName "kube-api-access-499r9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:40:13.794266 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.794224    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:13.857595 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.857529    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-499r9\" (UniqueName: \"kubernetes.io/projected/1170928a-f7e9-4b2a-bc57-106d01bdaf88-kube-api-access-499r9\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:40:13.857595 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.857587    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1170928a-f7e9-4b2a-bc57-106d01bdaf88-openshift-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:40:13.857595 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.857604    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1170928a-f7e9-4b2a-bc57-106d01bdaf88-maas-api-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:40:13.857888 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.857617    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1170928a-f7e9-4b2a-bc57-106d01bdaf88-metrics-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:40:13.868600 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:13.868544    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:14.182272 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:14.182222    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:14.281862 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:14.281829    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:14.390653 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:14.390619    2566 generic.go:356] "Generic (PLEG): container finished" podID="1170928a-f7e9-4b2a-bc57-106d01bdaf88" containerID="479c6c6e707d5df003e8910fecf6cac479bd26b19a0a5a70a040c804f8fc805e" exitCode=0
Sep 10 17:40:14.390826 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:14.390669    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"
Sep 10 17:40:14.390826 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:14.390705    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq" event={"ID":"1170928a-f7e9-4b2a-bc57-106d01bdaf88","Type":"ContainerDied","Data":"479c6c6e707d5df003e8910fecf6cac479bd26b19a0a5a70a040c804f8fc805e"}
Sep 10 17:40:14.390826 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:14.390756    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq" event={"ID":"1170928a-f7e9-4b2a-bc57-106d01bdaf88","Type":"ContainerDied","Data":"3dbdb74490b126e6099ea129fb9fd88e74cf2d2d3134e35a9adff21461ddbec0"}
Sep 10 17:40:14.390826 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:14.390781    2566 scope.go:117] "RemoveContainer" containerID="479c6c6e707d5df003e8910fecf6cac479bd26b19a0a5a70a040c804f8fc805e"
Sep 10 17:40:14.403022 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:14.403000    2566 scope.go:117] "RemoveContainer" containerID="479c6c6e707d5df003e8910fecf6cac479bd26b19a0a5a70a040c804f8fc805e"
Sep 10 17:40:14.403438 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:40:14.403413    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"479c6c6e707d5df003e8910fecf6cac479bd26b19a0a5a70a040c804f8fc805e\": container with ID starting with 479c6c6e707d5df003e8910fecf6cac479bd26b19a0a5a70a040c804f8fc805e not found: ID does not exist" containerID="479c6c6e707d5df003e8910fecf6cac479bd26b19a0a5a70a040c804f8fc805e"
Sep 10 17:40:14.403517 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:14.403446    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"479c6c6e707d5df003e8910fecf6cac479bd26b19a0a5a70a040c804f8fc805e"} err="failed to get container status \"479c6c6e707d5df003e8910fecf6cac479bd26b19a0a5a70a040c804f8fc805e\": rpc error: code = NotFound desc = could not find container \"479c6c6e707d5df003e8910fecf6cac479bd26b19a0a5a70a040c804f8fc805e\": container with ID starting with 479c6c6e707d5df003e8910fecf6cac479bd26b19a0a5a70a040c804f8fc805e not found: ID does not exist"
Sep 10 17:40:14.404916 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:14.404892    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:40:14.419079 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:14.419041    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"]
Sep 10 17:40:14.427020 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:14.426986    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-52f72f-5bd668f45f-qt2wq"]
Sep 10 17:40:14.835662 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:14.835628    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1170928a-f7e9-4b2a-bc57-106d01bdaf88" path="/var/lib/kubelet/pods/1170928a-f7e9-4b2a-bc57-106d01bdaf88/volumes"
Sep 10 17:40:15.154897 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:15.154794    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:16.011302 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:16.011260    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:16.861509 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:16.861473    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:17.150073 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:17.149987    2566 scope.go:117] "RemoveContainer" containerID="f4d83caec8b3afa24f625fe6d9f830c75b85d35b31be1d79c9060a2fd219d74a"
Sep 10 17:40:17.159486 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:17.159461    2566 scope.go:117] "RemoveContainer" containerID="a54f6402f4ed94d705cbbd28ee93974965f39f43e049f6658b923c61ccda3f82"
Sep 10 17:40:19.340511 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:19.340475    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:20.539444 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:20.539406    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:20.641434 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:20.641392    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:20.811362 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:20.811249    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:21.542737 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:21.542703    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:22.441020 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:22.440957    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:23.156289 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:23.156255    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:23.880607 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:23.880563    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:24.848823 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:24.848768    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:31.787886 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:31.787845    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"]
Sep 10 17:40:31.788405 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:31.788094    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj" podUID="c9ba7f62-2ff3-4263-a81e-c4ddc8e72870" containerName="maas-api" containerID="cri-o://ea70e68352ce5f0349a7280f3e4c2e0eed392b7d048ae0f61ca7e440cd381eb7" gracePeriod=30
Sep 10 17:40:31.815682 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:31.815643    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:31.890864 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:31.890820    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:31.957886 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:31.957815    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:32.030557 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.030523    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:32.049291 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.049226    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"
Sep 10 17:40:32.098980 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.098944    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:32.143081 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.143045    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-metrics-certs\") pod \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\" (UID: \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\") "
Sep 10 17:40:32.143081 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.143088    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-maas-api-tls\") pod \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\" (UID: \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\") "
Sep 10 17:40:32.143320 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.143207    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-openshift-service-ca\") pod \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\" (UID: \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\") "
Sep 10 17:40:32.143320 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.143243    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhkcg\" (UniqueName: \"kubernetes.io/projected/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-kube-api-access-fhkcg\") pod \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\" (UID: \"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870\") "
Sep 10 17:40:32.143583 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.143558    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c9ba7f62-2ff3-4263-a81e-c4ddc8e72870" (UID: "c9ba7f62-2ff3-4263-a81e-c4ddc8e72870"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:40:32.143682 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.143663    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-openshift-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:40:32.145380 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.145357    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c9ba7f62-2ff3-4263-a81e-c4ddc8e72870" (UID: "c9ba7f62-2ff3-4263-a81e-c4ddc8e72870"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:40:32.145461 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.145374    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-kube-api-access-fhkcg" (OuterVolumeSpecName: "kube-api-access-fhkcg") pod "c9ba7f62-2ff3-4263-a81e-c4ddc8e72870" (UID: "c9ba7f62-2ff3-4263-a81e-c4ddc8e72870"). InnerVolumeSpecName "kube-api-access-fhkcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:40:32.145461 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.145419    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c9ba7f62-2ff3-4263-a81e-c4ddc8e72870" (UID: "c9ba7f62-2ff3-4263-a81e-c4ddc8e72870"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:40:32.244975 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.244938    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-metrics-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:40:32.244975 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.244969    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-maas-api-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:40:32.244975 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.244977    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhkcg\" (UniqueName: \"kubernetes.io/projected/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870-kube-api-access-fhkcg\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:40:32.411481 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.411369    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:32.506766 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.506729    2566 generic.go:356] "Generic (PLEG): container finished" podID="c9ba7f62-2ff3-4263-a81e-c4ddc8e72870" containerID="ea70e68352ce5f0349a7280f3e4c2e0eed392b7d048ae0f61ca7e440cd381eb7" exitCode=0
Sep 10 17:40:32.506960 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.506781    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"
Sep 10 17:40:32.506960 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.506781    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj" event={"ID":"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870","Type":"ContainerDied","Data":"ea70e68352ce5f0349a7280f3e4c2e0eed392b7d048ae0f61ca7e440cd381eb7"}
Sep 10 17:40:32.506960 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.506840    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj" event={"ID":"c9ba7f62-2ff3-4263-a81e-c4ddc8e72870","Type":"ContainerDied","Data":"cfd94ed6d55bec37f0c59a1e083f90f0510624e2a6b3bf1d2ae16531ae648120"}
Sep 10 17:40:32.506960 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.506862    2566 scope.go:117] "RemoveContainer" containerID="ea70e68352ce5f0349a7280f3e4c2e0eed392b7d048ae0f61ca7e440cd381eb7"
Sep 10 17:40:32.517640 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.517614    2566 scope.go:117] "RemoveContainer" containerID="ea70e68352ce5f0349a7280f3e4c2e0eed392b7d048ae0f61ca7e440cd381eb7"
Sep 10 17:40:32.518028 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:40:32.518002    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea70e68352ce5f0349a7280f3e4c2e0eed392b7d048ae0f61ca7e440cd381eb7\": container with ID starting with ea70e68352ce5f0349a7280f3e4c2e0eed392b7d048ae0f61ca7e440cd381eb7 not found: ID does not exist" containerID="ea70e68352ce5f0349a7280f3e4c2e0eed392b7d048ae0f61ca7e440cd381eb7"
Sep 10 17:40:32.518149 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.518036    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea70e68352ce5f0349a7280f3e4c2e0eed392b7d048ae0f61ca7e440cd381eb7"} err="failed to get container status \"ea70e68352ce5f0349a7280f3e4c2e0eed392b7d048ae0f61ca7e440cd381eb7\": rpc error: code = NotFound desc = could not find container \"ea70e68352ce5f0349a7280f3e4c2e0eed392b7d048ae0f61ca7e440cd381eb7\": container with ID starting with ea70e68352ce5f0349a7280f3e4c2e0eed392b7d048ae0f61ca7e440cd381eb7 not found: ID does not exist"
Sep 10 17:40:32.524213 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.524182    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:32.530371 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.530348    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"]
Sep 10 17:40:32.534604 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.534568    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5912a3-867cfc7775-rg4vj"]
Sep 10 17:40:32.835693 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:32.835654    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9ba7f62-2ff3-4263-a81e-c4ddc8e72870" path="/var/lib/kubelet/pods/c9ba7f62-2ff3-4263-a81e-c4ddc8e72870/volumes"
Sep 10 17:40:33.346292 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:33.346252    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:34.073478 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:34.073437    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:35.946623 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:35.946583    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:38.595774 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:38.595723    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:38.667232 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:38.667191    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:38.873320 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:38.873213    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:50.043511 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:40:50.043465    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:12.893293 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:12.893257    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:17.511284 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:17.511244    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"]
Sep 10 17:41:17.521824 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:17.521779    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:18.463598 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:18.463560    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:18.860722 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:18.860685    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:19.199574 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:19.199469    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:19.739286 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:19.739248    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:21.773666 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:21.773627    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:22.448879 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:22.448834    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:22.545869 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:22.545832    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:22.699618 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:22.699523    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:23.607294 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:23.607261    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:24.511089 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.511033    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"]
Sep 10 17:41:24.511710 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.511684    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9ba7f62-2ff3-4263-a81e-c4ddc8e72870" containerName="maas-api"
Sep 10 17:41:24.511710 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.511707    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9ba7f62-2ff3-4263-a81e-c4ddc8e72870" containerName="maas-api"
Sep 10 17:41:24.511917 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.511747    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1170928a-f7e9-4b2a-bc57-106d01bdaf88" containerName="maas-api"
Sep 10 17:41:24.511917 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.511756    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1170928a-f7e9-4b2a-bc57-106d01bdaf88" containerName="maas-api"
Sep 10 17:41:24.511917 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.511846    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1170928a-f7e9-4b2a-bc57-106d01bdaf88" containerName="maas-api"
Sep 10 17:41:24.511917 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.511864    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9ba7f62-2ff3-4263-a81e-c4ddc8e72870" containerName="maas-api"
Sep 10 17:41:24.516614 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.516572    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"
Sep 10 17:41:24.519225 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.519201    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-7858f0-serving-cert\""
Sep 10 17:41:24.529403 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.529373    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"]
Sep 10 17:41:24.552050 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.552008    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:24.625496 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.625454    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:24.675783 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.675732    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58c4f377-452a-4321-9342-e77a85be0891-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd\" (UID: \"58c4f377-452a-4321-9342-e77a85be0891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"
Sep 10 17:41:24.675972 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.675789    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58c4f377-452a-4321-9342-e77a85be0891-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd\" (UID: \"58c4f377-452a-4321-9342-e77a85be0891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"
Sep 10 17:41:24.675972 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.675853    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58c4f377-452a-4321-9342-e77a85be0891-metrics-certs\") pod \"maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd\" (UID: \"58c4f377-452a-4321-9342-e77a85be0891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"
Sep 10 17:41:24.675972 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.675896    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw2wk\" (UniqueName: \"kubernetes.io/projected/58c4f377-452a-4321-9342-e77a85be0891-kube-api-access-sw2wk\") pod \"maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd\" (UID: \"58c4f377-452a-4321-9342-e77a85be0891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"
Sep 10 17:41:24.727418 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.727378    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:24.776856 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.776754    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58c4f377-452a-4321-9342-e77a85be0891-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd\" (UID: \"58c4f377-452a-4321-9342-e77a85be0891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"
Sep 10 17:41:24.776856 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.776809    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58c4f377-452a-4321-9342-e77a85be0891-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd\" (UID: \"58c4f377-452a-4321-9342-e77a85be0891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"
Sep 10 17:41:24.776856 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.776827    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58c4f377-452a-4321-9342-e77a85be0891-metrics-certs\") pod \"maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd\" (UID: \"58c4f377-452a-4321-9342-e77a85be0891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"
Sep 10 17:41:24.776856 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.776851    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw2wk\" (UniqueName: \"kubernetes.io/projected/58c4f377-452a-4321-9342-e77a85be0891-kube-api-access-sw2wk\") pod \"maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd\" (UID: \"58c4f377-452a-4321-9342-e77a85be0891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"
Sep 10 17:41:24.777631 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.777603    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58c4f377-452a-4321-9342-e77a85be0891-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd\" (UID: \"58c4f377-452a-4321-9342-e77a85be0891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"
Sep 10 17:41:24.779409 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.779381    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58c4f377-452a-4321-9342-e77a85be0891-metrics-certs\") pod \"maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd\" (UID: \"58c4f377-452a-4321-9342-e77a85be0891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"
Sep 10 17:41:24.779500 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.779438    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58c4f377-452a-4321-9342-e77a85be0891-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd\" (UID: \"58c4f377-452a-4321-9342-e77a85be0891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"
Sep 10 17:41:24.785606 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.785585    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw2wk\" (UniqueName: \"kubernetes.io/projected/58c4f377-452a-4321-9342-e77a85be0891-kube-api-access-sw2wk\") pod \"maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd\" (UID: \"58c4f377-452a-4321-9342-e77a85be0891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"
Sep 10 17:41:24.830361 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:24.830328    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"
Sep 10 17:41:25.052820 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:25.052724    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:25.116603 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:25.116562    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"]
Sep 10 17:41:25.118694 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:41:25.118653    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58c4f377_452a_4321_9342_e77a85be0891.slice/crio-86756dd88723dac4340fc61890e6d2a963d5080f2493a35fe272adbc49918a8d WatchSource:0}: Error finding container 86756dd88723dac4340fc61890e6d2a963d5080f2493a35fe272adbc49918a8d: Status 404 returned error can't find the container with id 86756dd88723dac4340fc61890e6d2a963d5080f2493a35fe272adbc49918a8d
Sep 10 17:41:25.123687 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:25.123649    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:25.437269 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:25.437168    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:25.834899 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:25.834857    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd" event={"ID":"58c4f377-452a-4321-9342-e77a85be0891","Type":"ContainerStarted","Data":"7562a4355120817e8205ffc20ab68f97e43c57be3bdc77682806ab66b94a7104"}
Sep 10 17:41:25.835359 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:25.834924    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd" event={"ID":"58c4f377-452a-4321-9342-e77a85be0891","Type":"ContainerStarted","Data":"86756dd88723dac4340fc61890e6d2a963d5080f2493a35fe272adbc49918a8d"}
Sep 10 17:41:25.835359 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:25.835047    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"
Sep 10 17:41:25.854983 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:25.854921    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd" podStartSLOduration=1.46923786 podStartE2EDuration="1.854898714s" totalImagesPullingTime="385.660854ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:25.852489298 +0000 UTC m=+1689.533662299" watchObservedRunningTime="2026-09-10 17:41:25.854898714 +0000 UTC m=+1689.536071658"
Sep 10 17:41:31.846709 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:31.846671    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"
Sep 10 17:41:32.531888 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.531842    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5" podUID="3505f047-2a6a-48c8-b243-d722453c4d80" containerName="main" containerID="cri-o://b5c3ef26153b3ac62af03eb9f530d47396ae9470967e404909d764cd7027d5a0" gracePeriod=45
Sep 10 17:41:32.788343 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.788273    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:41:32.864012 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.863971    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7dwf\" (UniqueName: \"kubernetes.io/projected/3505f047-2a6a-48c8-b243-d722453c4d80-kube-api-access-f7dwf\") pod \"3505f047-2a6a-48c8-b243-d722453c4d80\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") "
Sep 10 17:41:32.864543 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.864040    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-dshm\") pod \"3505f047-2a6a-48c8-b243-d722453c4d80\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") "
Sep 10 17:41:32.864543 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.864073    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3505f047-2a6a-48c8-b243-d722453c4d80-tls-certs\") pod \"3505f047-2a6a-48c8-b243-d722453c4d80\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") "
Sep 10 17:41:32.864543 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.864096    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-model-cache\") pod \"3505f047-2a6a-48c8-b243-d722453c4d80\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") "
Sep 10 17:41:32.864543 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.864179    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-home\") pod \"3505f047-2a6a-48c8-b243-d722453c4d80\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") "
Sep 10 17:41:32.864543 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.864219    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-tmp-dir\") pod \"3505f047-2a6a-48c8-b243-d722453c4d80\" (UID: \"3505f047-2a6a-48c8-b243-d722453c4d80\") "
Sep 10 17:41:32.864543 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.864486    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-home" (OuterVolumeSpecName: "home") pod "3505f047-2a6a-48c8-b243-d722453c4d80" (UID: "3505f047-2a6a-48c8-b243-d722453c4d80"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:41:32.864543 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.864503    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-model-cache" (OuterVolumeSpecName: "model-cache") pod "3505f047-2a6a-48c8-b243-d722453c4d80" (UID: "3505f047-2a6a-48c8-b243-d722453c4d80"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:41:32.864959 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.864599    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3505f047-2a6a-48c8-b243-d722453c4d80" (UID: "3505f047-2a6a-48c8-b243-d722453c4d80"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:41:32.866359 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.866333    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-dshm" (OuterVolumeSpecName: "dshm") pod "3505f047-2a6a-48c8-b243-d722453c4d80" (UID: "3505f047-2a6a-48c8-b243-d722453c4d80"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:41:32.866468 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.866362    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3505f047-2a6a-48c8-b243-d722453c4d80-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3505f047-2a6a-48c8-b243-d722453c4d80" (UID: "3505f047-2a6a-48c8-b243-d722453c4d80"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:41:32.866468 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.866370    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3505f047-2a6a-48c8-b243-d722453c4d80-kube-api-access-f7dwf" (OuterVolumeSpecName: "kube-api-access-f7dwf") pod "3505f047-2a6a-48c8-b243-d722453c4d80" (UID: "3505f047-2a6a-48c8-b243-d722453c4d80"). InnerVolumeSpecName "kube-api-access-f7dwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:41:32.878760 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.878730    2566 generic.go:356] "Generic (PLEG): container finished" podID="3505f047-2a6a-48c8-b243-d722453c4d80" containerID="b5c3ef26153b3ac62af03eb9f530d47396ae9470967e404909d764cd7027d5a0" exitCode=0
Sep 10 17:41:32.878904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.878776    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"
Sep 10 17:41:32.878904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.878811    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5" event={"ID":"3505f047-2a6a-48c8-b243-d722453c4d80","Type":"ContainerDied","Data":"b5c3ef26153b3ac62af03eb9f530d47396ae9470967e404909d764cd7027d5a0"}
Sep 10 17:41:32.878904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.878858    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5" event={"ID":"3505f047-2a6a-48c8-b243-d722453c4d80","Type":"ContainerDied","Data":"a7068fab434d97ff33269b22618e549f946917fd392c5404d3b72c25b71d37fd"}
Sep 10 17:41:32.878904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.878882    2566 scope.go:117] "RemoveContainer" containerID="b5c3ef26153b3ac62af03eb9f530d47396ae9470967e404909d764cd7027d5a0"
Sep 10 17:41:32.889976 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.889953    2566 scope.go:117] "RemoveContainer" containerID="b5c3ef26153b3ac62af03eb9f530d47396ae9470967e404909d764cd7027d5a0"
Sep 10 17:41:32.890417 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:41:32.890395    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5c3ef26153b3ac62af03eb9f530d47396ae9470967e404909d764cd7027d5a0\": container with ID starting with b5c3ef26153b3ac62af03eb9f530d47396ae9470967e404909d764cd7027d5a0 not found: ID does not exist" containerID="b5c3ef26153b3ac62af03eb9f530d47396ae9470967e404909d764cd7027d5a0"
Sep 10 17:41:32.890519 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.890425    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5c3ef26153b3ac62af03eb9f530d47396ae9470967e404909d764cd7027d5a0"} err="failed to get container status \"b5c3ef26153b3ac62af03eb9f530d47396ae9470967e404909d764cd7027d5a0\": rpc error: code = NotFound desc = could not find container \"b5c3ef26153b3ac62af03eb9f530d47396ae9470967e404909d764cd7027d5a0\": container with ID starting with b5c3ef26153b3ac62af03eb9f530d47396ae9470967e404909d764cd7027d5a0 not found: ID does not exist"
Sep 10 17:41:32.903381 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.903342    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"]
Sep 10 17:41:32.909814 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.909783    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3c30a2/rate-test-model-3c30a2-kserve-99474996c-nj7h5"]
Sep 10 17:41:32.965084 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.965048    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-tmp-dir\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:41:32.965084 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.965078    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7dwf\" (UniqueName: \"kubernetes.io/projected/3505f047-2a6a-48c8-b243-d722453c4d80-kube-api-access-f7dwf\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:41:32.965084 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.965089    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-dshm\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:41:32.965343 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.965099    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3505f047-2a6a-48c8-b243-d722453c4d80-tls-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:41:32.965343 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.965109    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-model-cache\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:41:32.965343 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:32.965116    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3505f047-2a6a-48c8-b243-d722453c4d80-home\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:41:34.835147 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:34.835081    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3505f047-2a6a-48c8-b243-d722453c4d80" path="/var/lib/kubelet/pods/3505f047-2a6a-48c8-b243-d722453c4d80/volumes"
Sep 10 17:41:46.841916 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:46.841820    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:47.479080 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:47.479038    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:47.746818 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:47.746779    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:48.219484 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:48.219381    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:48.573260 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:48.573220    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:48.721982 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:48.721930    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:48.858209 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:48.858085    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:49.865739 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:49.865696    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:50.955664 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:50.955629    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:51.732631 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:51.732592    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:51.848824 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:51.848786    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:52.862735 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:52.862698    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:53.728730 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:53.728688    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:54.614154 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.614088    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"]
Sep 10 17:41:54.614775 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.614751    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3505f047-2a6a-48c8-b243-d722453c4d80" containerName="main"
Sep 10 17:41:54.614875 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.614777    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3505f047-2a6a-48c8-b243-d722453c4d80" containerName="main"
Sep 10 17:41:54.614931 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.614875    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3505f047-2a6a-48c8-b243-d722453c4d80" containerName="main"
Sep 10 17:41:54.618396 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.618364    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"
Sep 10 17:41:54.621345 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.621319    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-fdb1bd-serving-cert\""
Sep 10 17:41:54.627902 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.627804    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"]
Sep 10 17:41:54.654752 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.654687    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:54.767607 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.767558    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:54.789815 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.789776    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aee87c9d-28c7-46bd-b88b-90a00043d753-maas-api-tls\") pod \"maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg\" (UID: \"aee87c9d-28c7-46bd-b88b-90a00043d753\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"
Sep 10 17:41:54.790200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.789920    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee87c9d-28c7-46bd-b88b-90a00043d753-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg\" (UID: \"aee87c9d-28c7-46bd-b88b-90a00043d753\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"
Sep 10 17:41:54.790200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.789958    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aee87c9d-28c7-46bd-b88b-90a00043d753-metrics-certs\") pod \"maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg\" (UID: \"aee87c9d-28c7-46bd-b88b-90a00043d753\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"
Sep 10 17:41:54.790200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.789981    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghnbp\" (UniqueName: \"kubernetes.io/projected/aee87c9d-28c7-46bd-b88b-90a00043d753-kube-api-access-ghnbp\") pod \"maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg\" (UID: \"aee87c9d-28c7-46bd-b88b-90a00043d753\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"
Sep 10 17:41:54.891066 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.890967    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee87c9d-28c7-46bd-b88b-90a00043d753-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg\" (UID: \"aee87c9d-28c7-46bd-b88b-90a00043d753\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"
Sep 10 17:41:54.891066 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.891012    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aee87c9d-28c7-46bd-b88b-90a00043d753-metrics-certs\") pod \"maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg\" (UID: \"aee87c9d-28c7-46bd-b88b-90a00043d753\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"
Sep 10 17:41:54.891066 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.891032    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghnbp\" (UniqueName: \"kubernetes.io/projected/aee87c9d-28c7-46bd-b88b-90a00043d753-kube-api-access-ghnbp\") pod \"maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg\" (UID: \"aee87c9d-28c7-46bd-b88b-90a00043d753\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"
Sep 10 17:41:54.891359 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.891072    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aee87c9d-28c7-46bd-b88b-90a00043d753-maas-api-tls\") pod \"maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg\" (UID: \"aee87c9d-28c7-46bd-b88b-90a00043d753\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"
Sep 10 17:41:54.891775 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.891743    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee87c9d-28c7-46bd-b88b-90a00043d753-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg\" (UID: \"aee87c9d-28c7-46bd-b88b-90a00043d753\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"
Sep 10 17:41:54.893692 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.893666    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aee87c9d-28c7-46bd-b88b-90a00043d753-metrics-certs\") pod \"maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg\" (UID: \"aee87c9d-28c7-46bd-b88b-90a00043d753\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"
Sep 10 17:41:54.893692 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.893680    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aee87c9d-28c7-46bd-b88b-90a00043d753-maas-api-tls\") pod \"maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg\" (UID: \"aee87c9d-28c7-46bd-b88b-90a00043d753\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"
Sep 10 17:41:54.900629 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.900603    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghnbp\" (UniqueName: \"kubernetes.io/projected/aee87c9d-28c7-46bd-b88b-90a00043d753-kube-api-access-ghnbp\") pod \"maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg\" (UID: \"aee87c9d-28c7-46bd-b88b-90a00043d753\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"
Sep 10 17:41:54.901981 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.901951    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:54.932908 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:54.932862    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"
Sep 10 17:41:55.233106 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:55.233016    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"]
Sep 10 17:41:55.236869 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:41:55.236839    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaee87c9d_28c7_46bd_b88b_90a00043d753.slice/crio-607845bdbb11900b1f69a8f13fbcd340ffbde257bd07772f445ec5ae9578b868 WatchSource:0}: Error finding container 607845bdbb11900b1f69a8f13fbcd340ffbde257bd07772f445ec5ae9578b868: Status 404 returned error can't find the container with id 607845bdbb11900b1f69a8f13fbcd340ffbde257bd07772f445ec5ae9578b868
Sep 10 17:41:55.286882 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:55.286845    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:55.396814 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:55.396760    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:56.029841 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:56.029802    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg" event={"ID":"aee87c9d-28c7-46bd-b88b-90a00043d753","Type":"ContainerStarted","Data":"2588e41289cf3d4963c258ff640258111a6d506013c849bdd330bcfb15e754fb"}
Sep 10 17:41:56.030310 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:56.029868    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg" event={"ID":"aee87c9d-28c7-46bd-b88b-90a00043d753","Type":"ContainerStarted","Data":"607845bdbb11900b1f69a8f13fbcd340ffbde257bd07772f445ec5ae9578b868"}
Sep 10 17:41:56.030310 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:56.030020    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"
Sep 10 17:41:56.051089 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:41:56.051035    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg" podStartSLOduration=1.725045532 podStartE2EDuration="2.051017623s" totalImagesPullingTime="325.972091ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:56.049216114 +0000 UTC m=+1719.730389061" watchObservedRunningTime="2026-09-10 17:41:56.051017623 +0000 UTC m=+1719.732190567"
Sep 10 17:42:02.042756 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:42:02.042724    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"
Sep 10 17:42:16.682577 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:42:16.682531    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:16.978365 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:42:16.978267    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:18.072904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:42:18.072869    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:18.963731 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:42:18.963692    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:19.723636 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:42:19.723573    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:20.553178 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:42:20.553143    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:27.553952 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:42:27.553913    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:32.605842 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:42:32.605797    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:58.327709 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:42:58.327669    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:58.690357 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:42:58.690254    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:59.292224 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:42:59.292179    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:01.356647 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:01.356586    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:02.053968 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:02.053936    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:02.153267 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:02.153226    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:02.274653 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:02.274604    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:03.262889 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:03.262853    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:04.042796 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:04.042752    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:04.774722 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:04.774679    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:05.709498 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:05.709457    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"]
Sep 10 17:43:05.713782 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:05.713752    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:05.716366 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:05.716340    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-8a34df-serving-cert\""
Sep 10 17:43:05.727799 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:05.726973    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"]
Sep 10 17:43:05.747200 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:05.747166    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:05.812558 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:05.812520    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54\" (UID: \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:05.813018 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:05.812590    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdf5t\" (UniqueName: \"kubernetes.io/projected/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-kube-api-access-fdf5t\") pod \"maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54\" (UID: \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:05.813018 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:05.812664    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54\" (UID: \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:05.813018 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:05.812743    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54\" (UID: \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:05.820480 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:05.820442    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:05.914228 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:05.914189    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdf5t\" (UniqueName: \"kubernetes.io/projected/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-kube-api-access-fdf5t\") pod \"maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54\" (UID: \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:05.914445 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:05.914263    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54\" (UID: \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:05.914445 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:05.914314    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54\" (UID: \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:05.914564 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:05.914483    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54\" (UID: \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:05.914658 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:43:05.914638    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-serving-cert: secret "maas-api-e2e-ipp-cleanup-8a34df-serving-cert" not found
Sep 10 17:43:05.914751 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:43:05.914719    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-maas-api-tls podName:96c74c84-19f3-4fa1-b5f4-4c5681ab663a nodeName:}" failed. No retries permitted until 2026-09-10 17:43:06.414697912 +0000 UTC m=+1790.095870835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54" (UID: "96c74c84-19f3-4fa1-b5f4-4c5681ab663a") : secret "maas-api-e2e-ipp-cleanup-8a34df-serving-cert" not found
Sep 10 17:43:05.915627 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:05.915598    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54\" (UID: \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:05.917324 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:05.917297    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54\" (UID: \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:05.923977 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:05.923943    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdf5t\" (UniqueName: \"kubernetes.io/projected/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-kube-api-access-fdf5t\") pod \"maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54\" (UID: \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:06.025203 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:06.025119    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:06.335255 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:06.335160    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:06.421018 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:06.420978    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54\" (UID: \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:06.423613 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:06.423592    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54\" (UID: \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:06.434232 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:06.434191    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:06.626767 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:06.626650    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:06.904714 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:06.904686    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"]
Sep 10 17:43:06.906237 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:43:06.906200    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96c74c84_19f3_4fa1_b5f4_4c5681ab663a.slice/crio-8a7c794ee2d35cceea084035545aed2143eedc75f45b0e11f878b18d33411e18 WatchSource:0}: Error finding container 8a7c794ee2d35cceea084035545aed2143eedc75f45b0e11f878b18d33411e18: Status 404 returned error can't find the container with id 8a7c794ee2d35cceea084035545aed2143eedc75f45b0e11f878b18d33411e18
Sep 10 17:43:07.477886 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:07.477835    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54" event={"ID":"96c74c84-19f3-4fa1-b5f4-4c5681ab663a","Type":"ContainerStarted","Data":"fef84100bbb096d9be758c0e933b1c4cc109cee9ea56c13b24e5b9b5aa618af6"}
Sep 10 17:43:07.478095 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:07.477900    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54" event={"ID":"96c74c84-19f3-4fa1-b5f4-4c5681ab663a","Type":"ContainerStarted","Data":"8a7c794ee2d35cceea084035545aed2143eedc75f45b0e11f878b18d33411e18"}
Sep 10 17:43:07.478095 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:07.477939    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:07.503564 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:07.503508    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54" podStartSLOduration=2.140583669 podStartE2EDuration="2.503488841s" totalImagesPullingTime="362.905172ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:43:07.501610883 +0000 UTC m=+1791.182783823" watchObservedRunningTime="2026-09-10 17:43:07.503488841 +0000 UTC m=+1791.184661784"
Sep 10 17:43:13.493416 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:13.493325    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:29.189821 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:29.189778    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:30.049759 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:30.049721    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:30.776609 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:30.776560    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:31.548525 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:31.548481    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:32.244211 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:32.244158    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:32.345773 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:32.345734    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:33.205764 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:33.205724    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:34.141793 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:34.141743    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:34.902570 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:34.902532    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:35.729512 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:35.729474    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:36.428925 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:36.428884    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:36.836346 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:36.836315    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:37.242034 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:37.241993    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:38.058721 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:38.058672    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:44.017002 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.016960    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"]
Sep 10 17:43:44.017535 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.017278    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54" podUID="96c74c84-19f3-4fa1-b5f4-4c5681ab663a" containerName="maas-api" containerID="cri-o://fef84100bbb096d9be758c0e933b1c4cc109cee9ea56c13b24e5b9b5aa618af6" gracePeriod=30
Sep 10 17:43:44.047910 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.047867    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:44.146744 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.146709    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:44.248902 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.248861    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:44.276511 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.276440    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:44.307499 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.307464    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdf5t\" (UniqueName: \"kubernetes.io/projected/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-kube-api-access-fdf5t\") pod \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\" (UID: \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\") "
Sep 10 17:43:44.307687 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.307523    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-maas-api-tls\") pod \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\" (UID: \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\") "
Sep 10 17:43:44.307687 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.307586    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-metrics-certs\") pod \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\" (UID: \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\") "
Sep 10 17:43:44.307687 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.307635    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-openshift-service-ca\") pod \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\" (UID: \"96c74c84-19f3-4fa1-b5f4-4c5681ab663a\") "
Sep 10 17:43:44.308072 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.308042    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "96c74c84-19f3-4fa1-b5f4-4c5681ab663a" (UID: "96c74c84-19f3-4fa1-b5f4-4c5681ab663a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:43:44.309696 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.309666    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-kube-api-access-fdf5t" (OuterVolumeSpecName: "kube-api-access-fdf5t") pod "96c74c84-19f3-4fa1-b5f4-4c5681ab663a" (UID: "96c74c84-19f3-4fa1-b5f4-4c5681ab663a"). InnerVolumeSpecName "kube-api-access-fdf5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:43:44.310181 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.310159    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "96c74c84-19f3-4fa1-b5f4-4c5681ab663a" (UID: "96c74c84-19f3-4fa1-b5f4-4c5681ab663a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:43:44.310251 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.310178    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "96c74c84-19f3-4fa1-b5f4-4c5681ab663a" (UID: "96c74c84-19f3-4fa1-b5f4-4c5681ab663a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:43:44.349975 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.349932    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:44.409446 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.409399    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-maas-api-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:43:44.409446 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.409444    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-metrics-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:43:44.409446 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.409457    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-openshift-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:43:44.409836 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.409466    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdf5t\" (UniqueName: \"kubernetes.io/projected/96c74c84-19f3-4fa1-b5f4-4c5681ab663a-kube-api-access-fdf5t\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:43:44.683151 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.683030    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:44.719449 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.719410    2566 generic.go:356] "Generic (PLEG): container finished" podID="96c74c84-19f3-4fa1-b5f4-4c5681ab663a" containerID="fef84100bbb096d9be758c0e933b1c4cc109cee9ea56c13b24e5b9b5aa618af6" exitCode=0
Sep 10 17:43:44.719449 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.719451    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54" event={"ID":"96c74c84-19f3-4fa1-b5f4-4c5681ab663a","Type":"ContainerDied","Data":"fef84100bbb096d9be758c0e933b1c4cc109cee9ea56c13b24e5b9b5aa618af6"}
Sep 10 17:43:44.719718 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.719468    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"
Sep 10 17:43:44.719718 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.719499    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54" event={"ID":"96c74c84-19f3-4fa1-b5f4-4c5681ab663a","Type":"ContainerDied","Data":"8a7c794ee2d35cceea084035545aed2143eedc75f45b0e11f878b18d33411e18"}
Sep 10 17:43:44.719718 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.719524    2566 scope.go:117] "RemoveContainer" containerID="fef84100bbb096d9be758c0e933b1c4cc109cee9ea56c13b24e5b9b5aa618af6"
Sep 10 17:43:44.730470 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.730447    2566 scope.go:117] "RemoveContainer" containerID="fef84100bbb096d9be758c0e933b1c4cc109cee9ea56c13b24e5b9b5aa618af6"
Sep 10 17:43:44.730801 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:43:44.730783    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fef84100bbb096d9be758c0e933b1c4cc109cee9ea56c13b24e5b9b5aa618af6\": container with ID starting with fef84100bbb096d9be758c0e933b1c4cc109cee9ea56c13b24e5b9b5aa618af6 not found: ID does not exist" containerID="fef84100bbb096d9be758c0e933b1c4cc109cee9ea56c13b24e5b9b5aa618af6"
Sep 10 17:43:44.730849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.730810    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fef84100bbb096d9be758c0e933b1c4cc109cee9ea56c13b24e5b9b5aa618af6"} err="failed to get container status \"fef84100bbb096d9be758c0e933b1c4cc109cee9ea56c13b24e5b9b5aa618af6\": rpc error: code = NotFound desc = could not find container \"fef84100bbb096d9be758c0e933b1c4cc109cee9ea56c13b24e5b9b5aa618af6\": container with ID starting with fef84100bbb096d9be758c0e933b1c4cc109cee9ea56c13b24e5b9b5aa618af6 not found: ID does not exist"
Sep 10 17:43:44.745269 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.745234    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"]
Sep 10 17:43:44.749490 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.749461    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a34df-765f784d76-rcg54"]
Sep 10 17:43:44.783651 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.783614    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:44.837549 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:44.837509    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96c74c84-19f3-4fa1-b5f4-4c5681ab663a" path="/var/lib/kubelet/pods/96c74c84-19f3-4fa1-b5f4-4c5681ab663a/volumes"
Sep 10 17:43:45.717134 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:45.717087    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:46.540881 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:46.540835    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:47.340601 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:47.340552    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:48.044968 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:48.044930    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:50.689784 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:50.689742    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:50.794398 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:50.794360    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:50.993503 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:50.993465    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:54.863937 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:54.863841    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:55.687713 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:55.687673    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:56.500311 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:56.500276    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:57.571355 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:57.571307    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:58.271372 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:58.271329    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:59.138295 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:43:59.138253    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:10.220765 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.220722    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"]
Sep 10 17:44:10.221272 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.221003    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd" podUID="58c4f377-452a-4321-9342-e77a85be0891" containerName="maas-api" containerID="cri-o://7562a4355120817e8205ffc20ab68f97e43c57be3bdc77682806ab66b94a7104" gracePeriod=30
Sep 10 17:44:10.237729 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.237682    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:10.315901 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.315862    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:10.386251 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.386215    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:10.462052 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.462003    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:10.490975 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.490943    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"
Sep 10 17:44:10.535117 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.535074    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:10.570372 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.570328    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sw2wk\" (UniqueName: \"kubernetes.io/projected/58c4f377-452a-4321-9342-e77a85be0891-kube-api-access-sw2wk\") pod \"58c4f377-452a-4321-9342-e77a85be0891\" (UID: \"58c4f377-452a-4321-9342-e77a85be0891\") "
Sep 10 17:44:10.570535 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.570408    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58c4f377-452a-4321-9342-e77a85be0891-maas-api-tls\") pod \"58c4f377-452a-4321-9342-e77a85be0891\" (UID: \"58c4f377-452a-4321-9342-e77a85be0891\") "
Sep 10 17:44:10.570535 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.570480    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58c4f377-452a-4321-9342-e77a85be0891-metrics-certs\") pod \"58c4f377-452a-4321-9342-e77a85be0891\" (UID: \"58c4f377-452a-4321-9342-e77a85be0891\") "
Sep 10 17:44:10.570535 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.570518    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58c4f377-452a-4321-9342-e77a85be0891-openshift-service-ca\") pod \"58c4f377-452a-4321-9342-e77a85be0891\" (UID: \"58c4f377-452a-4321-9342-e77a85be0891\") "
Sep 10 17:44:10.570977 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.570947    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58c4f377-452a-4321-9342-e77a85be0891-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "58c4f377-452a-4321-9342-e77a85be0891" (UID: "58c4f377-452a-4321-9342-e77a85be0891"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:44:10.572745 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.572704    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58c4f377-452a-4321-9342-e77a85be0891-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "58c4f377-452a-4321-9342-e77a85be0891" (UID: "58c4f377-452a-4321-9342-e77a85be0891"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:44:10.572862 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.572707    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58c4f377-452a-4321-9342-e77a85be0891-kube-api-access-sw2wk" (OuterVolumeSpecName: "kube-api-access-sw2wk") pod "58c4f377-452a-4321-9342-e77a85be0891" (UID: "58c4f377-452a-4321-9342-e77a85be0891"). InnerVolumeSpecName "kube-api-access-sw2wk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:44:10.572862 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.572719    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58c4f377-452a-4321-9342-e77a85be0891-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "58c4f377-452a-4321-9342-e77a85be0891" (UID: "58c4f377-452a-4321-9342-e77a85be0891"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:44:10.671825 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.671764    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sw2wk\" (UniqueName: \"kubernetes.io/projected/58c4f377-452a-4321-9342-e77a85be0891-kube-api-access-sw2wk\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:44:10.671825 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.671816    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58c4f377-452a-4321-9342-e77a85be0891-maas-api-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:44:10.671825 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.671826    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58c4f377-452a-4321-9342-e77a85be0891-metrics-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:44:10.671825 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.671835    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58c4f377-452a-4321-9342-e77a85be0891-openshift-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:44:10.864922 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.864884    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:10.880768 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.880729    2566 generic.go:356] "Generic (PLEG): container finished" podID="58c4f377-452a-4321-9342-e77a85be0891" containerID="7562a4355120817e8205ffc20ab68f97e43c57be3bdc77682806ab66b94a7104" exitCode=0
Sep 10 17:44:10.880971 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.880778    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"
Sep 10 17:44:10.880971 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.880812    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd" event={"ID":"58c4f377-452a-4321-9342-e77a85be0891","Type":"ContainerDied","Data":"7562a4355120817e8205ffc20ab68f97e43c57be3bdc77682806ab66b94a7104"}
Sep 10 17:44:10.880971 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.880884    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd" event={"ID":"58c4f377-452a-4321-9342-e77a85be0891","Type":"ContainerDied","Data":"86756dd88723dac4340fc61890e6d2a963d5080f2493a35fe272adbc49918a8d"}
Sep 10 17:44:10.880971 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.880918    2566 scope.go:117] "RemoveContainer" containerID="7562a4355120817e8205ffc20ab68f97e43c57be3bdc77682806ab66b94a7104"
Sep 10 17:44:10.890660 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.890637    2566 scope.go:117] "RemoveContainer" containerID="7562a4355120817e8205ffc20ab68f97e43c57be3bdc77682806ab66b94a7104"
Sep 10 17:44:10.890964 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:44:10.890944    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7562a4355120817e8205ffc20ab68f97e43c57be3bdc77682806ab66b94a7104\": container with ID starting with 7562a4355120817e8205ffc20ab68f97e43c57be3bdc77682806ab66b94a7104 not found: ID does not exist" containerID="7562a4355120817e8205ffc20ab68f97e43c57be3bdc77682806ab66b94a7104"
Sep 10 17:44:10.891047 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.890972    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7562a4355120817e8205ffc20ab68f97e43c57be3bdc77682806ab66b94a7104"} err="failed to get container status \"7562a4355120817e8205ffc20ab68f97e43c57be3bdc77682806ab66b94a7104\": rpc error: code = NotFound desc = could not find container \"7562a4355120817e8205ffc20ab68f97e43c57be3bdc77682806ab66b94a7104\": container with ID starting with 7562a4355120817e8205ffc20ab68f97e43c57be3bdc77682806ab66b94a7104 not found: ID does not exist"
Sep 10 17:44:10.897579 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.897551    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"]
Sep 10 17:44:10.902464 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.902441    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7858f0-75d9cc9949-h24rd"]
Sep 10 17:44:10.968040 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:10.967993    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:11.670530 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:11.670494    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:12.517701 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:12.517665    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:12.835310 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:12.835210    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58c4f377-452a-4321-9342-e77a85be0891" path="/var/lib/kubelet/pods/58c4f377-452a-4321-9342-e77a85be0891/volumes"
Sep 10 17:44:13.206466 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:13.206325    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:15.893792 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:15.893753    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:28.437366 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:28.437327    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:34.234674 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:34.234629    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:52.327631 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:52.327531    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:53.184946 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:53.184905    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:58.755411 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:58.755355    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:58.855601 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:58.855564    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:59.078801 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:59.078709    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:59.624695 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:44:59.624657    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:00.487254 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:00.487211    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:01.344907 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:01.344863    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:02.058789 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:02.058752    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:02.865832 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:02.865797    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:09.955785 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:09.955740    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"]
Sep 10 17:45:09.956353 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:09.955983    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg" podUID="aee87c9d-28c7-46bd-b88b-90a00043d753" containerName="maas-api" containerID="cri-o://2588e41289cf3d4963c258ff640258111a6d506013c849bdd330bcfb15e754fb" gracePeriod=30
Sep 10 17:45:09.971350 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:09.971294    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:10.050406 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.050366    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:10.142339 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.142205    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:10.217767 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.217681    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:10.222041 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.222015    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"
Sep 10 17:45:10.265068 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.265024    2566 generic.go:356] "Generic (PLEG): container finished" podID="aee87c9d-28c7-46bd-b88b-90a00043d753" containerID="2588e41289cf3d4963c258ff640258111a6d506013c849bdd330bcfb15e754fb" exitCode=0
Sep 10 17:45:10.265068 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.265069    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"
Sep 10 17:45:10.265068 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.265079    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg" event={"ID":"aee87c9d-28c7-46bd-b88b-90a00043d753","Type":"ContainerDied","Data":"2588e41289cf3d4963c258ff640258111a6d506013c849bdd330bcfb15e754fb"}
Sep 10 17:45:10.265351 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.265159    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg" event={"ID":"aee87c9d-28c7-46bd-b88b-90a00043d753","Type":"ContainerDied","Data":"607845bdbb11900b1f69a8f13fbcd340ffbde257bd07772f445ec5ae9578b868"}
Sep 10 17:45:10.265351 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.265196    2566 scope.go:117] "RemoveContainer" containerID="2588e41289cf3d4963c258ff640258111a6d506013c849bdd330bcfb15e754fb"
Sep 10 17:45:10.275204 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.275180    2566 scope.go:117] "RemoveContainer" containerID="2588e41289cf3d4963c258ff640258111a6d506013c849bdd330bcfb15e754fb"
Sep 10 17:45:10.275571 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:45:10.275550    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2588e41289cf3d4963c258ff640258111a6d506013c849bdd330bcfb15e754fb\": container with ID starting with 2588e41289cf3d4963c258ff640258111a6d506013c849bdd330bcfb15e754fb not found: ID does not exist" containerID="2588e41289cf3d4963c258ff640258111a6d506013c849bdd330bcfb15e754fb"
Sep 10 17:45:10.275626 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.275585    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2588e41289cf3d4963c258ff640258111a6d506013c849bdd330bcfb15e754fb"} err="failed to get container status \"2588e41289cf3d4963c258ff640258111a6d506013c849bdd330bcfb15e754fb\": rpc error: code = NotFound desc = could not find container \"2588e41289cf3d4963c258ff640258111a6d506013c849bdd330bcfb15e754fb\": container with ID starting with 2588e41289cf3d4963c258ff640258111a6d506013c849bdd330bcfb15e754fb not found: ID does not exist"
Sep 10 17:45:10.276874 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.276856    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aee87c9d-28c7-46bd-b88b-90a00043d753-metrics-certs\") pod \"aee87c9d-28c7-46bd-b88b-90a00043d753\" (UID: \"aee87c9d-28c7-46bd-b88b-90a00043d753\") "
Sep 10 17:45:10.276929 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.276892    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghnbp\" (UniqueName: \"kubernetes.io/projected/aee87c9d-28c7-46bd-b88b-90a00043d753-kube-api-access-ghnbp\") pod \"aee87c9d-28c7-46bd-b88b-90a00043d753\" (UID: \"aee87c9d-28c7-46bd-b88b-90a00043d753\") "
Sep 10 17:45:10.276929 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.276910    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aee87c9d-28c7-46bd-b88b-90a00043d753-maas-api-tls\") pod \"aee87c9d-28c7-46bd-b88b-90a00043d753\" (UID: \"aee87c9d-28c7-46bd-b88b-90a00043d753\") "
Sep 10 17:45:10.277018 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.276953    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee87c9d-28c7-46bd-b88b-90a00043d753-openshift-service-ca\") pod \"aee87c9d-28c7-46bd-b88b-90a00043d753\" (UID: \"aee87c9d-28c7-46bd-b88b-90a00043d753\") "
Sep 10 17:45:10.277368 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.277345    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aee87c9d-28c7-46bd-b88b-90a00043d753-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aee87c9d-28c7-46bd-b88b-90a00043d753" (UID: "aee87c9d-28c7-46bd-b88b-90a00043d753"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:45:10.279086 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.279057    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aee87c9d-28c7-46bd-b88b-90a00043d753-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aee87c9d-28c7-46bd-b88b-90a00043d753" (UID: "aee87c9d-28c7-46bd-b88b-90a00043d753"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:45:10.279086 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.279072    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aee87c9d-28c7-46bd-b88b-90a00043d753-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "aee87c9d-28c7-46bd-b88b-90a00043d753" (UID: "aee87c9d-28c7-46bd-b88b-90a00043d753"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:45:10.279262 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.279153    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aee87c9d-28c7-46bd-b88b-90a00043d753-kube-api-access-ghnbp" (OuterVolumeSpecName: "kube-api-access-ghnbp") pod "aee87c9d-28c7-46bd-b88b-90a00043d753" (UID: "aee87c9d-28c7-46bd-b88b-90a00043d753"). InnerVolumeSpecName "kube-api-access-ghnbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:45:10.286388 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.286361    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:10.377785 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.377748    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aee87c9d-28c7-46bd-b88b-90a00043d753-metrics-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:45:10.377785 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.377779    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghnbp\" (UniqueName: \"kubernetes.io/projected/aee87c9d-28c7-46bd-b88b-90a00043d753-kube-api-access-ghnbp\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:45:10.377785 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.377790    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aee87c9d-28c7-46bd-b88b-90a00043d753-maas-api-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:45:10.378045 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.377802    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee87c9d-28c7-46bd-b88b-90a00043d753-openshift-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:45:10.588544 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.588504    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"]
Sep 10 17:45:10.596814 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.596779    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fdb1bd-57c9f787d7-ssspg"]
Sep 10 17:45:10.634846 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.634807    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:10.734904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.734870    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:10.835056 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:10.834972    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aee87c9d-28c7-46bd-b88b-90a00043d753" path="/var/lib/kubelet/pods/aee87c9d-28c7-46bd-b88b-90a00043d753/volumes"
Sep 10 17:45:11.558801 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:11.558755    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:12.431668 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:12.431634    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:16.524996 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:16.524955    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:16.624575 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:16.624540    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:16.823707 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:16.823606    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:17.971932 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:17.971894    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:18.841138 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:18.841095    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:19.625753 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:19.625714    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:20.455877 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:20.455834    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:27.447768 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:27.447724    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:28.156006 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.155972    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"]
Sep 10 17:45:28.156261 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.156226    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn" podUID="bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b" containerName="maas-api" containerID="cri-o://f625fad7914ad08086aa18c732fa306a5b33c18db61104b2b34f11a141a32ea7" gracePeriod=30
Sep 10 17:45:28.188039 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.187988    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:28.256435 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.256397    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:28.319594 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.319551    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:28.374918 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.374885    2566 generic.go:356] "Generic (PLEG): container finished" podID="bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b" containerID="f625fad7914ad08086aa18c732fa306a5b33c18db61104b2b34f11a141a32ea7" exitCode=0
Sep 10 17:45:28.375113 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.374959    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn" event={"ID":"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b","Type":"ContainerDied","Data":"f625fad7914ad08086aa18c732fa306a5b33c18db61104b2b34f11a141a32ea7"}
Sep 10 17:45:28.397990 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.397947    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:28.417672 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.417604    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"
Sep 10 17:45:28.467454 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.467348    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:28.566557 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.566519    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-metrics-certs\") pod \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\" (UID: \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\") "
Sep 10 17:45:28.566775 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.566573    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-maas-api-tls\") pod \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\" (UID: \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\") "
Sep 10 17:45:28.566775 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.566683    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slsxg\" (UniqueName: \"kubernetes.io/projected/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-kube-api-access-slsxg\") pod \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\" (UID: \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\") "
Sep 10 17:45:28.566775 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.566723    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-openshift-service-ca\") pod \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\" (UID: \"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b\") "
Sep 10 17:45:28.567294 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.567255    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b" (UID: "bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:45:28.568932 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.568897    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b" (UID: "bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:45:28.568932 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.568916    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-kube-api-access-slsxg" (OuterVolumeSpecName: "kube-api-access-slsxg") pod "bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b" (UID: "bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b"). InnerVolumeSpecName "kube-api-access-slsxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:45:28.568932 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.568917    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b" (UID: "bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:45:28.668167 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.668057    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-openshift-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:45:28.668167 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.668090    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-metrics-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:45:28.668167 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.668101    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-maas-api-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:45:28.668167 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.668110    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slsxg\" (UniqueName: \"kubernetes.io/projected/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b-kube-api-access-slsxg\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:45:28.823021 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.822978    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:28.923881 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:28.923779    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:29.382578 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:29.382543    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"
Sep 10 17:45:29.382578 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:29.382572    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn" event={"ID":"bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b","Type":"ContainerDied","Data":"085da2ae86c7a1b906ddac1ecc700911622234a8b87a0ca46f3a6ea8e681d8a3"}
Sep 10 17:45:29.382852 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:29.382638    2566 scope.go:117] "RemoveContainer" containerID="f625fad7914ad08086aa18c732fa306a5b33c18db61104b2b34f11a141a32ea7"
Sep 10 17:45:29.406706 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:29.406671    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"]
Sep 10 17:45:29.412604 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:29.412570    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3c30a2-85684fbb6b-4tjzn"]
Sep 10 17:45:29.650899 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:29.650800    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:30.835247 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:30.835214    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b" path="/var/lib/kubelet/pods/bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b/volumes"
Sep 10 17:45:35.345643 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:35.345597    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:35.436641 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:35.436595    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:35.669016 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:35.668920    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:36.874624 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:36.874588    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:37.645000 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:37.644960    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:46.515366 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.515324    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"]
Sep 10 17:45:46.515827 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.515623    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz" podUID="9b387931-990a-4093-b866-c8bc2d4d0f8b" containerName="maas-api" containerID="cri-o://8136f4b40c957859cc46acf651f00278059ece17d29dc4df0006f6da0b04b4ee" gracePeriod=30
Sep 10 17:45:46.548831 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.548794    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:46.647848 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.647814    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:46.747172 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.747111    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:46.779292 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.779222    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:45:46.836649 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.836618    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b387931-990a-4093-b866-c8bc2d4d0f8b-metrics-certs\") pod \"9b387931-990a-4093-b866-c8bc2d4d0f8b\" (UID: \"9b387931-990a-4093-b866-c8bc2d4d0f8b\") "
Sep 10 17:45:46.836649 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.836651    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b387931-990a-4093-b866-c8bc2d4d0f8b-maas-api-tls\") pod \"9b387931-990a-4093-b866-c8bc2d4d0f8b\" (UID: \"9b387931-990a-4093-b866-c8bc2d4d0f8b\") "
Sep 10 17:45:46.836849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.836700    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bb27v\" (UniqueName: \"kubernetes.io/projected/9b387931-990a-4093-b866-c8bc2d4d0f8b-kube-api-access-bb27v\") pod \"9b387931-990a-4093-b866-c8bc2d4d0f8b\" (UID: \"9b387931-990a-4093-b866-c8bc2d4d0f8b\") "
Sep 10 17:45:46.836849 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.836802    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b387931-990a-4093-b866-c8bc2d4d0f8b-openshift-service-ca\") pod \"9b387931-990a-4093-b866-c8bc2d4d0f8b\" (UID: \"9b387931-990a-4093-b866-c8bc2d4d0f8b\") "
Sep 10 17:45:46.837232 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.837200    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b387931-990a-4093-b866-c8bc2d4d0f8b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9b387931-990a-4093-b866-c8bc2d4d0f8b" (UID: "9b387931-990a-4093-b866-c8bc2d4d0f8b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:45:46.838832 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.838792    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b387931-990a-4093-b866-c8bc2d4d0f8b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9b387931-990a-4093-b866-c8bc2d4d0f8b" (UID: "9b387931-990a-4093-b866-c8bc2d4d0f8b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:45:46.838832 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.838793    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b387931-990a-4093-b866-c8bc2d4d0f8b-kube-api-access-bb27v" (OuterVolumeSpecName: "kube-api-access-bb27v") pod "9b387931-990a-4093-b866-c8bc2d4d0f8b" (UID: "9b387931-990a-4093-b866-c8bc2d4d0f8b"). InnerVolumeSpecName "kube-api-access-bb27v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:45:46.838990 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.838843    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b387931-990a-4093-b866-c8bc2d4d0f8b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9b387931-990a-4093-b866-c8bc2d4d0f8b" (UID: "9b387931-990a-4093-b866-c8bc2d4d0f8b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:45:46.847512 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.847480    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:46.938144 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.938081    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb27v\" (UniqueName: \"kubernetes.io/projected/9b387931-990a-4093-b866-c8bc2d4d0f8b-kube-api-access-bb27v\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:45:46.938144 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.938115    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b387931-990a-4093-b866-c8bc2d4d0f8b-openshift-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:45:46.938144 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.938139    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b387931-990a-4093-b866-c8bc2d4d0f8b-metrics-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:45:46.938144 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:46.938148    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b387931-990a-4093-b866-c8bc2d4d0f8b-maas-api-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:45:47.173626 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:47.173520    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:47.274182 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:47.274112    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:47.494855 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:47.494824    2566 generic.go:356] "Generic (PLEG): container finished" podID="9b387931-990a-4093-b866-c8bc2d4d0f8b" containerID="8136f4b40c957859cc46acf651f00278059ece17d29dc4df0006f6da0b04b4ee" exitCode=0
Sep 10 17:45:47.495030 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:47.494866    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"
Sep 10 17:45:47.495030 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:47.494897    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz" event={"ID":"9b387931-990a-4093-b866-c8bc2d4d0f8b","Type":"ContainerDied","Data":"8136f4b40c957859cc46acf651f00278059ece17d29dc4df0006f6da0b04b4ee"}
Sep 10 17:45:47.495030 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:47.494944    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz" event={"ID":"9b387931-990a-4093-b866-c8bc2d4d0f8b","Type":"ContainerDied","Data":"a07c3f9989b566621f1b4534523902bf4f1d355da6d5628157266198347181ec"}
Sep 10 17:45:47.495030 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:47.494968    2566 scope.go:117] "RemoveContainer" containerID="8136f4b40c957859cc46acf651f00278059ece17d29dc4df0006f6da0b04b4ee"
Sep 10 17:45:47.504861 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:47.504839    2566 scope.go:117] "RemoveContainer" containerID="8136f4b40c957859cc46acf651f00278059ece17d29dc4df0006f6da0b04b4ee"
Sep 10 17:45:47.505236 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:45:47.505213    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8136f4b40c957859cc46acf651f00278059ece17d29dc4df0006f6da0b04b4ee\": container with ID starting with 8136f4b40c957859cc46acf651f00278059ece17d29dc4df0006f6da0b04b4ee not found: ID does not exist" containerID="8136f4b40c957859cc46acf651f00278059ece17d29dc4df0006f6da0b04b4ee"
Sep 10 17:45:47.505282 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:47.505247    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8136f4b40c957859cc46acf651f00278059ece17d29dc4df0006f6da0b04b4ee"} err="failed to get container status \"8136f4b40c957859cc46acf651f00278059ece17d29dc4df0006f6da0b04b4ee\": rpc error: code = NotFound desc = could not find container \"8136f4b40c957859cc46acf651f00278059ece17d29dc4df0006f6da0b04b4ee\": container with ID starting with 8136f4b40c957859cc46acf651f00278059ece17d29dc4df0006f6da0b04b4ee not found: ID does not exist"
Sep 10 17:45:47.518404 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:47.518370    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"]
Sep 10 17:45:47.524235 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:47.524204    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c1fb61-6555f66896-zp5cz"]
Sep 10 17:45:48.835116 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:48.835074    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b387931-990a-4093-b866-c8bc2d4d0f8b" path="/var/lib/kubelet/pods/9b387931-990a-4093-b866-c8bc2d4d0f8b/volumes"
Sep 10 17:45:53.089831 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:53.089787    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:53.189741 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:53.189689    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:53.385750 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:45:53.385664    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:46:41.646548 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:46:41.646512    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:46:53.223355 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:46:53.223312    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:47:14.520075 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:47:14.520026    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:47:21.602797 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:47:21.602759    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:47:23.026791 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:47:23.026753    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:47:39.608218 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:47:39.608099    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:47:39.708484 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:47:39.708443    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:30.948694 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:48:30.948651    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2"]
Sep 10 17:48:30.949257 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:48:30.948857    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2" podUID="0c5d9784-5fa1-48a8-a857-d7bcc4756c3a" containerName="manager" containerID="cri-o://386659ce699a98359c1f088771c6638c4f122cdddc56880aec4e94b895b17a8f" gracePeriod=10
Sep 10 17:48:31.466599 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:48:31.466561    2566 generic.go:356] "Generic (PLEG): container finished" podID="0c5d9784-5fa1-48a8-a857-d7bcc4756c3a" containerID="386659ce699a98359c1f088771c6638c4f122cdddc56880aec4e94b895b17a8f" exitCode=0
Sep 10 17:48:31.466803 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:48:31.466637    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2" event={"ID":"0c5d9784-5fa1-48a8-a857-d7bcc4756c3a","Type":"ContainerDied","Data":"386659ce699a98359c1f088771c6638c4f122cdddc56880aec4e94b895b17a8f"}
Sep 10 17:48:31.710164 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:48:31.710118    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2"
Sep 10 17:48:31.809504 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:48:31.809460    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-748rs\" (UniqueName: \"kubernetes.io/projected/0c5d9784-5fa1-48a8-a857-d7bcc4756c3a-kube-api-access-748rs\") pod \"0c5d9784-5fa1-48a8-a857-d7bcc4756c3a\" (UID: \"0c5d9784-5fa1-48a8-a857-d7bcc4756c3a\") "
Sep 10 17:48:31.809704 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:48:31.809527    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c5d9784-5fa1-48a8-a857-d7bcc4756c3a-extensions-socket-volume\") pod \"0c5d9784-5fa1-48a8-a857-d7bcc4756c3a\" (UID: \"0c5d9784-5fa1-48a8-a857-d7bcc4756c3a\") "
Sep 10 17:48:31.809994 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:48:31.809964    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c5d9784-5fa1-48a8-a857-d7bcc4756c3a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0c5d9784-5fa1-48a8-a857-d7bcc4756c3a" (UID: "0c5d9784-5fa1-48a8-a857-d7bcc4756c3a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:48:31.811781 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:48:31.811759    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c5d9784-5fa1-48a8-a857-d7bcc4756c3a-kube-api-access-748rs" (OuterVolumeSpecName: "kube-api-access-748rs") pod "0c5d9784-5fa1-48a8-a857-d7bcc4756c3a" (UID: "0c5d9784-5fa1-48a8-a857-d7bcc4756c3a"). InnerVolumeSpecName "kube-api-access-748rs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:48:31.911181 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:48:31.911103    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-748rs\" (UniqueName: \"kubernetes.io/projected/0c5d9784-5fa1-48a8-a857-d7bcc4756c3a-kube-api-access-748rs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:48:31.911181 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:48:31.911172    2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c5d9784-5fa1-48a8-a857-d7bcc4756c3a-extensions-socket-volume\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:48:32.474410 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:48:32.474370    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2" event={"ID":"0c5d9784-5fa1-48a8-a857-d7bcc4756c3a","Type":"ContainerDied","Data":"b593201d16849071eec1b7c0fb580873831d5d5007d7613c659793f6cfda8e46"}
Sep 10 17:48:32.474891 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:48:32.474444    2566 scope.go:117] "RemoveContainer" containerID="386659ce699a98359c1f088771c6638c4f122cdddc56880aec4e94b895b17a8f"
Sep 10 17:48:32.474891 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:48:32.474390    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2"
Sep 10 17:48:32.498307 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:48:32.498273    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2"]
Sep 10 17:48:32.503045 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:48:32.503015    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mnfx2"]
Sep 10 17:48:32.835000 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:48:32.834969    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c5d9784-5fa1-48a8-a857-d7bcc4756c3a" path="/var/lib/kubelet/pods/0c5d9784-5fa1-48a8-a857-d7bcc4756c3a/volumes"
Sep 10 17:48:56.199725 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:48:56.199683    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:31.644506 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.644464    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fr6j8"]
Sep 10 17:49:31.645023 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.644889    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c5d9784-5fa1-48a8-a857-d7bcc4756c3a" containerName="manager"
Sep 10 17:49:31.645023 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.644902    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c5d9784-5fa1-48a8-a857-d7bcc4756c3a" containerName="manager"
Sep 10 17:49:31.645023 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.644918    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b" containerName="maas-api"
Sep 10 17:49:31.645023 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.644923    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b" containerName="maas-api"
Sep 10 17:49:31.645023 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.644931    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58c4f377-452a-4321-9342-e77a85be0891" containerName="maas-api"
Sep 10 17:49:31.645023 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.644937    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="58c4f377-452a-4321-9342-e77a85be0891" containerName="maas-api"
Sep 10 17:49:31.645023 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.644947    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96c74c84-19f3-4fa1-b5f4-4c5681ab663a" containerName="maas-api"
Sep 10 17:49:31.645023 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.644952    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="96c74c84-19f3-4fa1-b5f4-4c5681ab663a" containerName="maas-api"
Sep 10 17:49:31.645023 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.644960    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aee87c9d-28c7-46bd-b88b-90a00043d753" containerName="maas-api"
Sep 10 17:49:31.645023 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.644965    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="aee87c9d-28c7-46bd-b88b-90a00043d753" containerName="maas-api"
Sep 10 17:49:31.645023 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.644972    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b387931-990a-4093-b866-c8bc2d4d0f8b" containerName="maas-api"
Sep 10 17:49:31.645023 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.644977    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b387931-990a-4093-b866-c8bc2d4d0f8b" containerName="maas-api"
Sep 10 17:49:31.645444 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.645044    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="58c4f377-452a-4321-9342-e77a85be0891" containerName="maas-api"
Sep 10 17:49:31.645444 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.645051    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf9242ef-2f31-4ba5-8dd6-0bf4dc75893b" containerName="maas-api"
Sep 10 17:49:31.645444 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.645057    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="aee87c9d-28c7-46bd-b88b-90a00043d753" containerName="maas-api"
Sep 10 17:49:31.645444 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.645062    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="96c74c84-19f3-4fa1-b5f4-4c5681ab663a" containerName="maas-api"
Sep 10 17:49:31.645444 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.645069    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c5d9784-5fa1-48a8-a857-d7bcc4756c3a" containerName="manager"
Sep 10 17:49:31.645444 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.645074    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b387931-990a-4093-b866-c8bc2d4d0f8b" containerName="maas-api"
Sep 10 17:49:31.648271 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.648254    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fr6j8"
Sep 10 17:49:31.653292 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.653268    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fpvh4\""
Sep 10 17:49:31.661519 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.661492    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fr6j8"]
Sep 10 17:49:31.685329 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.685289    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trm65\" (UniqueName: \"kubernetes.io/projected/065c2b96-3b7f-4987-8074-f8e67b449bf4-kube-api-access-trm65\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fr6j8\" (UID: \"065c2b96-3b7f-4987-8074-f8e67b449bf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fr6j8"
Sep 10 17:49:31.685515 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.685459    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/065c2b96-3b7f-4987-8074-f8e67b449bf4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fr6j8\" (UID: \"065c2b96-3b7f-4987-8074-f8e67b449bf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fr6j8"
Sep 10 17:49:31.786444 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.786401    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trm65\" (UniqueName: \"kubernetes.io/projected/065c2b96-3b7f-4987-8074-f8e67b449bf4-kube-api-access-trm65\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fr6j8\" (UID: \"065c2b96-3b7f-4987-8074-f8e67b449bf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fr6j8"
Sep 10 17:49:31.786655 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.786529    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/065c2b96-3b7f-4987-8074-f8e67b449bf4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fr6j8\" (UID: \"065c2b96-3b7f-4987-8074-f8e67b449bf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fr6j8"
Sep 10 17:49:31.786961 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.786934    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/065c2b96-3b7f-4987-8074-f8e67b449bf4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fr6j8\" (UID: \"065c2b96-3b7f-4987-8074-f8e67b449bf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fr6j8"
Sep 10 17:49:31.820615 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.820576    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trm65\" (UniqueName: \"kubernetes.io/projected/065c2b96-3b7f-4987-8074-f8e67b449bf4-kube-api-access-trm65\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fr6j8\" (UID: \"065c2b96-3b7f-4987-8074-f8e67b449bf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fr6j8"
Sep 10 17:49:31.959160 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:31.959051    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fr6j8"
Sep 10 17:49:32.254284 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:32.254259    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fr6j8"]
Sep 10 17:49:32.256500 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:49:32.256468    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod065c2b96_3b7f_4987_8074_f8e67b449bf4.slice/crio-aec8581d0ee3a457a1fc9b63f529593e54305d5451c6e7918105fa540b5a0264 WatchSource:0}: Error finding container aec8581d0ee3a457a1fc9b63f529593e54305d5451c6e7918105fa540b5a0264: Status 404 returned error can't find the container with id aec8581d0ee3a457a1fc9b63f529593e54305d5451c6e7918105fa540b5a0264
Sep 10 17:49:32.825983 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:32.825944    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fr6j8" event={"ID":"065c2b96-3b7f-4987-8074-f8e67b449bf4","Type":"ContainerStarted","Data":"005a48f61a090fc97fef550c27d2bc139d28b06092d533b84c184b9b6036e2bd"}
Sep 10 17:49:32.826409 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:32.826002    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fr6j8" event={"ID":"065c2b96-3b7f-4987-8074-f8e67b449bf4","Type":"ContainerStarted","Data":"aec8581d0ee3a457a1fc9b63f529593e54305d5451c6e7918105fa540b5a0264"}
Sep 10 17:49:32.826409 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:32.826073    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fr6j8"
Sep 10 17:49:32.848669 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:32.848613    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fr6j8" podStartSLOduration=1.848585637 podStartE2EDuration="1.848585637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:49:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:49:32.846380714 +0000 UTC m=+2176.527553679" watchObservedRunningTime="2026-09-10 17:49:32.848585637 +0000 UTC m=+2176.529758580"
Sep 10 17:49:43.834413 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:43.834378    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fr6j8"
Sep 10 17:49:51.847710 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:49:51.847665    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:11.540565 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:50:11.540523    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:15.601633 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:50:15.601584    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:20.849693 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:50:20.849643    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:56.541925 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:50:56.541887    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:57.258399 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:50:57.258355    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:51:16.432148 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.432092    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"]
Sep 10 17:51:16.435779 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.435753    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"
Sep 10 17:51:16.444526 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.444497    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 17:51:16.445569 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.445541    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 17:51:16.445718 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.445569    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:51:16.445718 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.445568    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-2hfkp\""
Sep 10 17:51:16.445718 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.445575    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 17:51:16.461763 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.461732    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"]
Sep 10 17:51:16.601976 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.601931    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41aa9888-57e2-4379-a6e8-f6e974915270-openshift-service-ca\") pod \"maas-api-6c68568df5-25xrz\" (UID: \"41aa9888-57e2-4379-a6e8-f6e974915270\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"
Sep 10 17:51:16.601976 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.601980    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrdz4\" (UniqueName: \"kubernetes.io/projected/41aa9888-57e2-4379-a6e8-f6e974915270-kube-api-access-mrdz4\") pod \"maas-api-6c68568df5-25xrz\" (UID: \"41aa9888-57e2-4379-a6e8-f6e974915270\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"
Sep 10 17:51:16.602267 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.602088    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41aa9888-57e2-4379-a6e8-f6e974915270-maas-api-tls\") pod \"maas-api-6c68568df5-25xrz\" (UID: \"41aa9888-57e2-4379-a6e8-f6e974915270\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"
Sep 10 17:51:16.602267 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.602172    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41aa9888-57e2-4379-a6e8-f6e974915270-metrics-certs\") pod \"maas-api-6c68568df5-25xrz\" (UID: \"41aa9888-57e2-4379-a6e8-f6e974915270\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"
Sep 10 17:51:16.703191 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.703091    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrdz4\" (UniqueName: \"kubernetes.io/projected/41aa9888-57e2-4379-a6e8-f6e974915270-kube-api-access-mrdz4\") pod \"maas-api-6c68568df5-25xrz\" (UID: \"41aa9888-57e2-4379-a6e8-f6e974915270\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"
Sep 10 17:51:16.703369 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.703201    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41aa9888-57e2-4379-a6e8-f6e974915270-maas-api-tls\") pod \"maas-api-6c68568df5-25xrz\" (UID: \"41aa9888-57e2-4379-a6e8-f6e974915270\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"
Sep 10 17:51:16.703369 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.703234    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41aa9888-57e2-4379-a6e8-f6e974915270-metrics-certs\") pod \"maas-api-6c68568df5-25xrz\" (UID: \"41aa9888-57e2-4379-a6e8-f6e974915270\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"
Sep 10 17:51:16.703369 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.703286    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41aa9888-57e2-4379-a6e8-f6e974915270-openshift-service-ca\") pod \"maas-api-6c68568df5-25xrz\" (UID: \"41aa9888-57e2-4379-a6e8-f6e974915270\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"
Sep 10 17:51:16.703891 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.703862    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41aa9888-57e2-4379-a6e8-f6e974915270-openshift-service-ca\") pod \"maas-api-6c68568df5-25xrz\" (UID: \"41aa9888-57e2-4379-a6e8-f6e974915270\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"
Sep 10 17:51:16.705778 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.705744    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41aa9888-57e2-4379-a6e8-f6e974915270-metrics-certs\") pod \"maas-api-6c68568df5-25xrz\" (UID: \"41aa9888-57e2-4379-a6e8-f6e974915270\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"
Sep 10 17:51:16.705889 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.705800    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41aa9888-57e2-4379-a6e8-f6e974915270-maas-api-tls\") pod \"maas-api-6c68568df5-25xrz\" (UID: \"41aa9888-57e2-4379-a6e8-f6e974915270\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"
Sep 10 17:51:16.714758 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.714728    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrdz4\" (UniqueName: \"kubernetes.io/projected/41aa9888-57e2-4379-a6e8-f6e974915270-kube-api-access-mrdz4\") pod \"maas-api-6c68568df5-25xrz\" (UID: \"41aa9888-57e2-4379-a6e8-f6e974915270\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"
Sep 10 17:51:16.756812 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.756778    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-2hfkp\""
Sep 10 17:51:16.764685 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:16.764656    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"
Sep 10 17:51:17.048824 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:17.048794    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"]
Sep 10 17:51:17.050552 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:51:17.050525    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41aa9888_57e2_4379_a6e8_f6e974915270.slice/crio-cb38f39d1901fe8110c7cd03eccd562c72a8a9692a99064f1c2f6694726464a9 WatchSource:0}: Error finding container cb38f39d1901fe8110c7cd03eccd562c72a8a9692a99064f1c2f6694726464a9: Status 404 returned error can't find the container with id cb38f39d1901fe8110c7cd03eccd562c72a8a9692a99064f1c2f6694726464a9
Sep 10 17:51:17.051958 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:17.051937    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 17:51:17.464397 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:17.464313    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz" event={"ID":"41aa9888-57e2-4379-a6e8-f6e974915270","Type":"ContainerStarted","Data":"cb38f39d1901fe8110c7cd03eccd562c72a8a9692a99064f1c2f6694726464a9"}
Sep 10 17:51:18.471434 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:18.471391    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz" event={"ID":"41aa9888-57e2-4379-a6e8-f6e974915270","Type":"ContainerStarted","Data":"5c61f206bee7e2d0734ee3a6b1bb1a239630f12c2234bc8be89fd317f69ca8de"}
Sep 10 17:51:18.471922 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:18.471459    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"
Sep 10 17:51:18.490858 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:18.490801    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz" podStartSLOduration=2.053150608 podStartE2EDuration="2.490784232s" totalImagesPullingTime="437.633624ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:51:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:51:18.488084318 +0000 UTC m=+2282.169257264" watchObservedRunningTime="2026-09-10 17:51:18.490784232 +0000 UTC m=+2282.171957174"
Sep 10 17:51:24.456404 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.456366    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"]
Sep 10 17:51:24.456960 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.456607    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz" podUID="41aa9888-57e2-4379-a6e8-f6e974915270" containerName="maas-api" containerID="cri-o://5c61f206bee7e2d0734ee3a6b1bb1a239630f12c2234bc8be89fd317f69ca8de" gracePeriod=30
Sep 10 17:51:24.462632 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.462605    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"
Sep 10 17:51:24.488226 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.488188    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"]
Sep 10 17:51:24.491921 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.491892    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"
Sep 10 17:51:24.502456 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.502422    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"]
Sep 10 17:51:24.580065 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.580033    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb-maas-api-tls\") pod \"maas-api-ddc7f4579-96sdn\" (UID: \"05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"
Sep 10 17:51:24.580292 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.580085    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45fw6\" (UniqueName: \"kubernetes.io/projected/05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb-kube-api-access-45fw6\") pod \"maas-api-ddc7f4579-96sdn\" (UID: \"05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"
Sep 10 17:51:24.580292 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.580263    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb-openshift-service-ca\") pod \"maas-api-ddc7f4579-96sdn\" (UID: \"05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"
Sep 10 17:51:24.580420 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.580388    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb-metrics-certs\") pod \"maas-api-ddc7f4579-96sdn\" (UID: \"05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"
Sep 10 17:51:24.680939 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.680898    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb-openshift-service-ca\") pod \"maas-api-ddc7f4579-96sdn\" (UID: \"05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"
Sep 10 17:51:24.681186 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.680975    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb-metrics-certs\") pod \"maas-api-ddc7f4579-96sdn\" (UID: \"05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"
Sep 10 17:51:24.681186 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.681018    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb-maas-api-tls\") pod \"maas-api-ddc7f4579-96sdn\" (UID: \"05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"
Sep 10 17:51:24.681186 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.681041    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45fw6\" (UniqueName: \"kubernetes.io/projected/05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb-kube-api-access-45fw6\") pod \"maas-api-ddc7f4579-96sdn\" (UID: \"05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"
Sep 10 17:51:24.681732 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.681708    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb-openshift-service-ca\") pod \"maas-api-ddc7f4579-96sdn\" (UID: \"05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"
Sep 10 17:51:24.683706 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.683678    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb-metrics-certs\") pod \"maas-api-ddc7f4579-96sdn\" (UID: \"05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"
Sep 10 17:51:24.683800 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.683763    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb-maas-api-tls\") pod \"maas-api-ddc7f4579-96sdn\" (UID: \"05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"
Sep 10 17:51:24.690284 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.690251    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45fw6\" (UniqueName: \"kubernetes.io/projected/05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb-kube-api-access-45fw6\") pod \"maas-api-ddc7f4579-96sdn\" (UID: \"05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"
Sep 10 17:51:24.714361 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.714289    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"
Sep 10 17:51:24.782310 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.782267    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41aa9888-57e2-4379-a6e8-f6e974915270-maas-api-tls\") pod \"41aa9888-57e2-4379-a6e8-f6e974915270\" (UID: \"41aa9888-57e2-4379-a6e8-f6e974915270\") "
Sep 10 17:51:24.782481 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.782389    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41aa9888-57e2-4379-a6e8-f6e974915270-openshift-service-ca\") pod \"41aa9888-57e2-4379-a6e8-f6e974915270\" (UID: \"41aa9888-57e2-4379-a6e8-f6e974915270\") "
Sep 10 17:51:24.782481 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.782443    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrdz4\" (UniqueName: \"kubernetes.io/projected/41aa9888-57e2-4379-a6e8-f6e974915270-kube-api-access-mrdz4\") pod \"41aa9888-57e2-4379-a6e8-f6e974915270\" (UID: \"41aa9888-57e2-4379-a6e8-f6e974915270\") "
Sep 10 17:51:24.782481 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.782467    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41aa9888-57e2-4379-a6e8-f6e974915270-metrics-certs\") pod \"41aa9888-57e2-4379-a6e8-f6e974915270\" (UID: \"41aa9888-57e2-4379-a6e8-f6e974915270\") "
Sep 10 17:51:24.782754 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.782727    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41aa9888-57e2-4379-a6e8-f6e974915270-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "41aa9888-57e2-4379-a6e8-f6e974915270" (UID: "41aa9888-57e2-4379-a6e8-f6e974915270"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:51:24.784753 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.784718    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41aa9888-57e2-4379-a6e8-f6e974915270-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "41aa9888-57e2-4379-a6e8-f6e974915270" (UID: "41aa9888-57e2-4379-a6e8-f6e974915270"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:51:24.784753 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.784729    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41aa9888-57e2-4379-a6e8-f6e974915270-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "41aa9888-57e2-4379-a6e8-f6e974915270" (UID: "41aa9888-57e2-4379-a6e8-f6e974915270"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:51:24.784891 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.784760    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41aa9888-57e2-4379-a6e8-f6e974915270-kube-api-access-mrdz4" (OuterVolumeSpecName: "kube-api-access-mrdz4") pod "41aa9888-57e2-4379-a6e8-f6e974915270" (UID: "41aa9888-57e2-4379-a6e8-f6e974915270"). InnerVolumeSpecName "kube-api-access-mrdz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:51:24.841448 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.841414    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"
Sep 10 17:51:24.884391 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.884351    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrdz4\" (UniqueName: \"kubernetes.io/projected/41aa9888-57e2-4379-a6e8-f6e974915270-kube-api-access-mrdz4\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:51:24.884391 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.884387    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41aa9888-57e2-4379-a6e8-f6e974915270-metrics-certs\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:51:24.884391 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.884401    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41aa9888-57e2-4379-a6e8-f6e974915270-maas-api-tls\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:51:24.884661 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:24.884412    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41aa9888-57e2-4379-a6e8-f6e974915270-openshift-service-ca\") on node \"ip-10-0-138-123.ec2.internal\" DevicePath \"\""
Sep 10 17:51:25.123965 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:25.123934    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"]
Sep 10 17:51:25.519931 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:25.519534    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn" event={"ID":"05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb","Type":"ContainerStarted","Data":"308ea790497df9c0aa79572e2f3022fb1d026b2654a06956bdc7de1bd949c76f"}
Sep 10 17:51:25.521612 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:25.521581    2566 generic.go:356] "Generic (PLEG): container finished" podID="41aa9888-57e2-4379-a6e8-f6e974915270" containerID="5c61f206bee7e2d0734ee3a6b1bb1a239630f12c2234bc8be89fd317f69ca8de" exitCode=0
Sep 10 17:51:25.521747 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:25.521644    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"
Sep 10 17:51:25.521747 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:25.521665    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz" event={"ID":"41aa9888-57e2-4379-a6e8-f6e974915270","Type":"ContainerDied","Data":"5c61f206bee7e2d0734ee3a6b1bb1a239630f12c2234bc8be89fd317f69ca8de"}
Sep 10 17:51:25.521747 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:25.521724    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c68568df5-25xrz" event={"ID":"41aa9888-57e2-4379-a6e8-f6e974915270","Type":"ContainerDied","Data":"cb38f39d1901fe8110c7cd03eccd562c72a8a9692a99064f1c2f6694726464a9"}
Sep 10 17:51:25.521927 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:25.521756    2566 scope.go:117] "RemoveContainer" containerID="5c61f206bee7e2d0734ee3a6b1bb1a239630f12c2234bc8be89fd317f69ca8de"
Sep 10 17:51:25.533238 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:25.533207    2566 scope.go:117] "RemoveContainer" containerID="5c61f206bee7e2d0734ee3a6b1bb1a239630f12c2234bc8be89fd317f69ca8de"
Sep 10 17:51:25.533559 ip-10-0-138-123 kubenswrapper[2566]: E0910 17:51:25.533538    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c61f206bee7e2d0734ee3a6b1bb1a239630f12c2234bc8be89fd317f69ca8de\": container with ID starting with 5c61f206bee7e2d0734ee3a6b1bb1a239630f12c2234bc8be89fd317f69ca8de not found: ID does not exist" containerID="5c61f206bee7e2d0734ee3a6b1bb1a239630f12c2234bc8be89fd317f69ca8de"
Sep 10 17:51:25.533603 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:25.533569    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c61f206bee7e2d0734ee3a6b1bb1a239630f12c2234bc8be89fd317f69ca8de"} err="failed to get container status \"5c61f206bee7e2d0734ee3a6b1bb1a239630f12c2234bc8be89fd317f69ca8de\": rpc error: code = NotFound desc = could not find container \"5c61f206bee7e2d0734ee3a6b1bb1a239630f12c2234bc8be89fd317f69ca8de\": container with ID starting with 5c61f206bee7e2d0734ee3a6b1bb1a239630f12c2234bc8be89fd317f69ca8de not found: ID does not exist"
Sep 10 17:51:25.539525 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:25.539494    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"]
Sep 10 17:51:25.545692 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:25.545665    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c68568df5-25xrz"]
Sep 10 17:51:26.530887 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:26.530848    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn" event={"ID":"05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb","Type":"ContainerStarted","Data":"b70bdf03d2af11535bea8e540aaa3c21bd4fe2fac822f10d58796d702420f8cd"}
Sep 10 17:51:26.531724 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:26.530921    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"
Sep 10 17:51:26.549094 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:26.549001    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn" podStartSLOduration=2.201606205 podStartE2EDuration="2.548978874s" totalImagesPullingTime="347.372669ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:51:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:51:26.548587664 +0000 UTC m=+2290.229760608" watchObservedRunningTime="2026-09-10 17:51:26.548978874 +0000 UTC m=+2290.230151818"
Sep 10 17:51:26.834644 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:26.834563    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41aa9888-57e2-4379-a6e8-f6e974915270" path="/var/lib/kubelet/pods/41aa9888-57e2-4379-a6e8-f6e974915270/volumes"
Sep 10 17:51:32.542898 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:32.542854    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-ddc7f4579-96sdn"
Sep 10 17:51:55.591937 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:55.591899    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-c7l6n_0c6b4d0c-ca19-4f6e-a766-05e618791c31/server/0.log"
Sep 10 17:51:55.716387 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:55.716343    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7bbsn_745988e5-d7ff-43e3-bf69-3e7372106216/manager/0.log"
Sep 10 17:51:55.829847 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:55.829814    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8dc495f6d-rt6q6_c4aa8ddd-f642-4ce0-b313-8684380b650f/copy-manifests/0.log"
Sep 10 17:51:55.849149 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:55.849036    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8dc495f6d-rt6q6_c4aa8ddd-f642-4ce0-b313-8684380b650f/manager/0.log"
Sep 10 17:51:56.435302 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:56.435268    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-ddc7f4579-96sdn_05b84fc4-fb3e-4d11-8fcf-f95d2a4095cb/maas-api/0.log"
Sep 10 17:51:57.810093 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:57.810055    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-2xlwn_65070453-dac7-4952-8ee1-be03f3968a8d/manager/0.log"
Sep 10 17:51:58.174438 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:58.174339    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-fr6j8_065c2b96-3b7f-4987-8074-f8e67b449bf4/manager/0.log"
Sep 10 17:51:58.419661 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:58.419628    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-mz88z_a4d4cbb7-a40b-4ba5-934a-bb988edc9741/manager/0.log"
Sep 10 17:51:58.762277 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:51:58.762246    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz_4daf4486-5c21-4743-ba60-484fbfc18dd9/istio-proxy/0.log"
Sep 10 17:52:00.325204 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:00.325173    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-98jjj_bff45ed2-316b-4c3e-8578-62156fdcb4fb/main/0.log"
Sep 10 17:52:06.437171 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:06.437112    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7ws5r/must-gather-btpcf"]
Sep 10 17:52:06.437781 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:06.437758    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41aa9888-57e2-4379-a6e8-f6e974915270" containerName="maas-api"
Sep 10 17:52:06.437883 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:06.437783    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="41aa9888-57e2-4379-a6e8-f6e974915270" containerName="maas-api"
Sep 10 17:52:06.437943 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:06.437902    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="41aa9888-57e2-4379-a6e8-f6e974915270" containerName="maas-api"
Sep 10 17:52:06.441949 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:06.441923    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7ws5r/must-gather-btpcf"
Sep 10 17:52:06.444719 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:06.444694    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7ws5r\"/\"openshift-service-ca.crt\""
Sep 10 17:52:06.445483 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:06.445464    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7ws5r\"/\"default-dockercfg-9c22d\""
Sep 10 17:52:06.445599 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:06.445513    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7ws5r\"/\"kube-root-ca.crt\""
Sep 10 17:52:06.450433 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:06.449929    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7ws5r/must-gather-btpcf"]
Sep 10 17:52:06.455192 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:06.455163    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:06.495529 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:06.495483    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d04b5b73-322f-4b89-8142-27ae4b86c32b-must-gather-output\") pod \"must-gather-btpcf\" (UID: \"d04b5b73-322f-4b89-8142-27ae4b86c32b\") " pod="openshift-must-gather-7ws5r/must-gather-btpcf"
Sep 10 17:52:06.495529 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:06.495528    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hszfm\" (UniqueName: \"kubernetes.io/projected/d04b5b73-322f-4b89-8142-27ae4b86c32b-kube-api-access-hszfm\") pod \"must-gather-btpcf\" (UID: \"d04b5b73-322f-4b89-8142-27ae4b86c32b\") " pod="openshift-must-gather-7ws5r/must-gather-btpcf"
Sep 10 17:52:06.529482 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:06.529443    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:06.596502 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:06.596462    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d04b5b73-322f-4b89-8142-27ae4b86c32b-must-gather-output\") pod \"must-gather-btpcf\" (UID: \"d04b5b73-322f-4b89-8142-27ae4b86c32b\") " pod="openshift-must-gather-7ws5r/must-gather-btpcf"
Sep 10 17:52:06.596502 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:06.596502    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hszfm\" (UniqueName: \"kubernetes.io/projected/d04b5b73-322f-4b89-8142-27ae4b86c32b-kube-api-access-hszfm\") pod \"must-gather-btpcf\" (UID: \"d04b5b73-322f-4b89-8142-27ae4b86c32b\") " pod="openshift-must-gather-7ws5r/must-gather-btpcf"
Sep 10 17:52:06.596860 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:06.596833    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d04b5b73-322f-4b89-8142-27ae4b86c32b-must-gather-output\") pod \"must-gather-btpcf\" (UID: \"d04b5b73-322f-4b89-8142-27ae4b86c32b\") " pod="openshift-must-gather-7ws5r/must-gather-btpcf"
Sep 10 17:52:06.604589 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:06.604553    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hszfm\" (UniqueName: \"kubernetes.io/projected/d04b5b73-322f-4b89-8142-27ae4b86c32b-kube-api-access-hszfm\") pod \"must-gather-btpcf\" (UID: \"d04b5b73-322f-4b89-8142-27ae4b86c32b\") " pod="openshift-must-gather-7ws5r/must-gather-btpcf"
Sep 10 17:52:06.753648 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:06.753613    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7ws5r/must-gather-btpcf"
Sep 10 17:52:07.031799 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:07.031766    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7ws5r/must-gather-btpcf"]
Sep 10 17:52:07.033594 ip-10-0-138-123 kubenswrapper[2566]: W0910 17:52:07.033568    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd04b5b73_322f_4b89_8142_27ae4b86c32b.slice/crio-c413c6221a35619c489c9b11ddd57a54d95d109ad62caa7391c8e82aa52d968a WatchSource:0}: Error finding container c413c6221a35619c489c9b11ddd57a54d95d109ad62caa7391c8e82aa52d968a: Status 404 returned error can't find the container with id c413c6221a35619c489c9b11ddd57a54d95d109ad62caa7391c8e82aa52d968a
Sep 10 17:52:07.782819 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:07.782775    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7ws5r/must-gather-btpcf" event={"ID":"d04b5b73-322f-4b89-8142-27ae4b86c32b","Type":"ContainerStarted","Data":"c413c6221a35619c489c9b11ddd57a54d95d109ad62caa7391c8e82aa52d968a"}
Sep 10 17:52:08.796024 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:08.795913    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7ws5r/must-gather-btpcf" event={"ID":"d04b5b73-322f-4b89-8142-27ae4b86c32b","Type":"ContainerStarted","Data":"67fcdea1975339bd683a5c121d399d457a4a9095323aaa2b5beda3a681afdc22"}
Sep 10 17:52:08.796024 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:08.795984    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7ws5r/must-gather-btpcf" event={"ID":"d04b5b73-322f-4b89-8142-27ae4b86c32b","Type":"ContainerStarted","Data":"64cd0db434898339709efb1e1503ab9bee48e5eba39882432363e5b3290e6b52"}
Sep 10 17:52:08.814676 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:08.814600    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7ws5r/must-gather-btpcf" podStartSLOduration=2.007676728 podStartE2EDuration="2.814576791s" totalImagesPullingTime="806.900063ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:52:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:52:08.811176972 +0000 UTC m=+2332.492349916" watchObservedRunningTime="2026-09-10 17:52:08.814576791 +0000 UTC m=+2332.495749761"
Sep 10 17:52:09.519210 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:09.519171    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qqw5f_5d278a9b-4d67-4eaf-a7a4-d2e980c2a775/global-pull-secret-syncer/0.log"
Sep 10 17:52:09.585137 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:09.585100    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log"
Sep 10 17:52:09.699904 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:09.699864    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-123.ec2.internal_b7691c42ef3bfce427afbddd33d4f4b3/haproxy/0.log"
Sep 10 17:52:15.135735 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:15.135694    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-2xlwn_65070453-dac7-4952-8ee1-be03f3968a8d/manager/0.log"
Sep 10 17:52:15.257086 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:15.257039    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-fr6j8_065c2b96-3b7f-4987-8074-f8e67b449bf4/manager/0.log"
Sep 10 17:52:15.363063 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:15.363024    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-mz88z_a4d4cbb7-a40b-4ba5-934a-bb988edc9741/manager/0.log"
Sep 10 17:52:16.935044 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:16.935003    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8c7a6e39-572d-432b-91cd-82f436f688c8/alertmanager/0.log"
Sep 10 17:52:16.969996 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:16.969960    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8c7a6e39-572d-432b-91cd-82f436f688c8/config-reloader/0.log"
Sep 10 17:52:17.019803 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:17.019767    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8c7a6e39-572d-432b-91cd-82f436f688c8/kube-rbac-proxy-web/0.log"
Sep 10 17:52:17.088946 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:17.088851    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8c7a6e39-572d-432b-91cd-82f436f688c8/kube-rbac-proxy/0.log"
Sep 10 17:52:17.116430 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:17.115188    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8c7a6e39-572d-432b-91cd-82f436f688c8/kube-rbac-proxy-metric/0.log"
Sep 10 17:52:17.144401 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:17.144361    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8c7a6e39-572d-432b-91cd-82f436f688c8/prom-label-proxy/0.log"
Sep 10 17:52:17.169715 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:17.169499    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8c7a6e39-572d-432b-91cd-82f436f688c8/init-config-reloader/0.log"
Sep 10 17:52:17.239174 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:17.239137    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-l4b56_bde2e973-b9b2-4b62-ad5d-1eb467ef1210/kube-state-metrics/0.log"
Sep 10 17:52:17.267884 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:17.267849    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-l4b56_bde2e973-b9b2-4b62-ad5d-1eb467ef1210/kube-rbac-proxy-main/0.log"
Sep 10 17:52:17.291266 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:17.291173    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-l4b56_bde2e973-b9b2-4b62-ad5d-1eb467ef1210/kube-rbac-proxy-self/0.log"
Sep 10 17:52:17.350602 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:17.350511    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-mkm8q_9fd7049a-7081-4356-9111-58beb92aa7b3/monitoring-plugin/0.log"
Sep 10 17:52:17.473108 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:17.473074    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9w7qm_30d419bb-93c9-465c-b86c-31291a08dd78/node-exporter/0.log"
Sep 10 17:52:17.498057 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:17.498026    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9w7qm_30d419bb-93c9-465c-b86c-31291a08dd78/kube-rbac-proxy/0.log"
Sep 10 17:52:17.519614 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:17.519583    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9w7qm_30d419bb-93c9-465c-b86c-31291a08dd78/init-textfile/0.log"
Sep 10 17:52:17.645972 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:17.645883    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-5txp5_baf56765-367e-4fb5-82ed-838f05e2818c/kube-rbac-proxy-main/0.log"
Sep 10 17:52:17.677763 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:17.677663    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-5txp5_baf56765-367e-4fb5-82ed-838f05e2818c/kube-rbac-proxy-self/0.log"
Sep 10 17:52:17.701652 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:17.701617    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-5txp5_baf56765-367e-4fb5-82ed-838f05e2818c/openshift-state-metrics/0.log"
Sep 10 17:52:17.909572 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:17.909490    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-pcljz_fb2e3d7c-ddea-4060-ba1b-4450b7522775/prometheus-operator/0.log"
Sep 10 17:52:17.926840 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:17.926805    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-pcljz_fb2e3d7c-ddea-4060-ba1b-4450b7522775/kube-rbac-proxy/0.log"
Sep 10 17:52:17.952110 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:17.952079    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-76wwx_e1226059-45ac-43b9-8a04-8c47c900178e/prometheus-operator-admission-webhook/0.log"
Sep 10 17:52:18.004642 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:18.004607    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7dff7cbc5-2fjcw_ff3911e9-03a4-4893-877a-795506a81a05/telemeter-client/0.log"
Sep 10 17:52:18.028992 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:18.028945    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7dff7cbc5-2fjcw_ff3911e9-03a4-4893-877a-795506a81a05/reload/0.log"
Sep 10 17:52:18.051275 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:18.051243    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7dff7cbc5-2fjcw_ff3911e9-03a4-4893-877a-795506a81a05/kube-rbac-proxy/0.log"
Sep 10 17:52:18.084082 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:18.084054    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bc4ff5bff-dcvn2_b743a223-6782-45ce-84c5-cefdfca6ed76/thanos-query/0.log"
Sep 10 17:52:18.107055 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:18.107015    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bc4ff5bff-dcvn2_b743a223-6782-45ce-84c5-cefdfca6ed76/kube-rbac-proxy-web/0.log"
Sep 10 17:52:18.134042 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:18.134008    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bc4ff5bff-dcvn2_b743a223-6782-45ce-84c5-cefdfca6ed76/kube-rbac-proxy/0.log"
Sep 10 17:52:18.157787 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:18.157736    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bc4ff5bff-dcvn2_b743a223-6782-45ce-84c5-cefdfca6ed76/prom-label-proxy/0.log"
Sep 10 17:52:18.180638 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:18.180538    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bc4ff5bff-dcvn2_b743a223-6782-45ce-84c5-cefdfca6ed76/kube-rbac-proxy-rules/0.log"
Sep 10 17:52:18.202664 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:18.202631    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bc4ff5bff-dcvn2_b743a223-6782-45ce-84c5-cefdfca6ed76/kube-rbac-proxy-metrics/0.log"
Sep 10 17:52:19.411426 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:19.411389    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-pvqls_367ec36e-c951-4ec0-9ddc-08af4f0f6714/networking-console-plugin/0.log"
Sep 10 17:52:20.156018 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.155973    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"]
Sep 10 17:52:20.164778 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.164742    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:20.170604 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.170570    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"]
Sep 10 17:52:20.262229 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.262188    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/221982c4-dc4d-4dfd-b243-5a54600931bc-lib-modules\") pod \"perf-node-gather-daemonset-rgxc5\" (UID: \"221982c4-dc4d-4dfd-b243-5a54600931bc\") " pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:20.262541 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.262517    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/221982c4-dc4d-4dfd-b243-5a54600931bc-podres\") pod \"perf-node-gather-daemonset-rgxc5\" (UID: \"221982c4-dc4d-4dfd-b243-5a54600931bc\") " pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:20.262691 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.262675    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/221982c4-dc4d-4dfd-b243-5a54600931bc-proc\") pod \"perf-node-gather-daemonset-rgxc5\" (UID: \"221982c4-dc4d-4dfd-b243-5a54600931bc\") " pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:20.262843 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.262827    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/221982c4-dc4d-4dfd-b243-5a54600931bc-sys\") pod \"perf-node-gather-daemonset-rgxc5\" (UID: \"221982c4-dc4d-4dfd-b243-5a54600931bc\") " pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:20.263028 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.263007    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxr6d\" (UniqueName: \"kubernetes.io/projected/221982c4-dc4d-4dfd-b243-5a54600931bc-kube-api-access-jxr6d\") pod \"perf-node-gather-daemonset-rgxc5\" (UID: \"221982c4-dc4d-4dfd-b243-5a54600931bc\") " pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:20.363938 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.363899    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/221982c4-dc4d-4dfd-b243-5a54600931bc-podres\") pod \"perf-node-gather-daemonset-rgxc5\" (UID: \"221982c4-dc4d-4dfd-b243-5a54600931bc\") " pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:20.363938 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.363940    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/221982c4-dc4d-4dfd-b243-5a54600931bc-proc\") pod \"perf-node-gather-daemonset-rgxc5\" (UID: \"221982c4-dc4d-4dfd-b243-5a54600931bc\") " pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:20.364279 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.363987    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/221982c4-dc4d-4dfd-b243-5a54600931bc-sys\") pod \"perf-node-gather-daemonset-rgxc5\" (UID: \"221982c4-dc4d-4dfd-b243-5a54600931bc\") " pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:20.364279 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.364059    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxr6d\" (UniqueName: \"kubernetes.io/projected/221982c4-dc4d-4dfd-b243-5a54600931bc-kube-api-access-jxr6d\") pod \"perf-node-gather-daemonset-rgxc5\" (UID: \"221982c4-dc4d-4dfd-b243-5a54600931bc\") " pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:20.364279 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.364112    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/221982c4-dc4d-4dfd-b243-5a54600931bc-podres\") pod \"perf-node-gather-daemonset-rgxc5\" (UID: \"221982c4-dc4d-4dfd-b243-5a54600931bc\") " pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:20.364279 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.364112    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/221982c4-dc4d-4dfd-b243-5a54600931bc-proc\") pod \"perf-node-gather-daemonset-rgxc5\" (UID: \"221982c4-dc4d-4dfd-b243-5a54600931bc\") " pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:20.364279 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.364150    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/221982c4-dc4d-4dfd-b243-5a54600931bc-lib-modules\") pod \"perf-node-gather-daemonset-rgxc5\" (UID: \"221982c4-dc4d-4dfd-b243-5a54600931bc\") " pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:20.364279 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.364196    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/221982c4-dc4d-4dfd-b243-5a54600931bc-sys\") pod \"perf-node-gather-daemonset-rgxc5\" (UID: \"221982c4-dc4d-4dfd-b243-5a54600931bc\") " pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:20.364279 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.364250    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/221982c4-dc4d-4dfd-b243-5a54600931bc-lib-modules\") pod \"perf-node-gather-daemonset-rgxc5\" (UID: \"221982c4-dc4d-4dfd-b243-5a54600931bc\") " pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:20.374052 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.374013    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxr6d\" (UniqueName: \"kubernetes.io/projected/221982c4-dc4d-4dfd-b243-5a54600931bc-kube-api-access-jxr6d\") pod \"perf-node-gather-daemonset-rgxc5\" (UID: \"221982c4-dc4d-4dfd-b243-5a54600931bc\") " pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:20.484161 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.484043    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:20.625544 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.625499    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6589f5969b-7gkhx_3db60130-0267-461f-9083-928fc93a1d4b/console/0.log"
Sep 10 17:52:20.666450 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.666418    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-5mpq7_e6d42206-f79c-43c6-bdc4-991cd18fd0cd/download-server/0.log"
Sep 10 17:52:20.798799 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.798737    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"]
Sep 10 17:52:20.911795 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:20.911755    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5" event={"ID":"221982c4-dc4d-4dfd-b243-5a54600931bc","Type":"ContainerStarted","Data":"2b1693def03d62445c3205aaedfe6bb4f109229efeacd928fbc09c143326133c"}
Sep 10 17:52:21.921644 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:21.921594    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5" event={"ID":"221982c4-dc4d-4dfd-b243-5a54600931bc","Type":"ContainerStarted","Data":"1f0d19caf93bd9209a18793f5bc731351b8a236b63a18e45dd93128a3f77c2d3"}
Sep 10 17:52:21.922679 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:21.922649    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:21.940574 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:21.940502    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5" podStartSLOduration=1.940481696 podStartE2EDuration="1.940481696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:52:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:52:21.937643267 +0000 UTC m=+2345.618816208" watchObservedRunningTime="2026-09-10 17:52:21.940481696 +0000 UTC m=+2345.621654639"
Sep 10 17:52:22.461476 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:22.461443    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pwhzb_2f11bf9c-45a3-4b27-8f8d-4a1d52999da2/dns/0.log"
Sep 10 17:52:22.484076 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:22.484047    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pwhzb_2f11bf9c-45a3-4b27-8f8d-4a1d52999da2/kube-rbac-proxy/0.log"
Sep 10 17:52:22.511936 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:22.511906    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fl4lm_69cd67eb-da77-4826-9bc2-a5202871c135/dns-node-resolver/0.log"
Sep 10 17:52:23.118363 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:23.118317    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-plvzg_26f57ed5-a830-4098-a2ae-651e7cd00567/node-ca/0.log"
Sep 10 17:52:24.196246 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:24.196214    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz_4daf4486-5c21-4743-ba60-484fbfc18dd9/istio-proxy/0.log"
Sep 10 17:52:25.607935 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:25.607900    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vkbjg_7cf62bc7-5bc1-4f6a-b73b-a7e488ab8c6f/serve-healthcheck-canary/0.log"
Sep 10 17:52:26.222450 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:26.222416    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6n9rh_53ecc34b-1bcb-4544-8c0c-a80d1c99ab04/kube-rbac-proxy/0.log"
Sep 10 17:52:26.243953 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:26.243920    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6n9rh_53ecc34b-1bcb-4544-8c0c-a80d1c99ab04/exporter/0.log"
Sep 10 17:52:26.266717 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:26.266672    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6n9rh_53ecc34b-1bcb-4544-8c0c-a80d1c99ab04/extractor/0.log"
Sep 10 17:52:28.796489 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:28.796431    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-c7l6n_0c6b4d0c-ca19-4f6e-a766-05e618791c31/server/0.log"
Sep 10 17:52:28.933080 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:28.933031    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7bbsn_745988e5-d7ff-43e3-bf69-3e7372106216/manager/0.log"
Sep 10 17:52:28.948954 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:28.948918    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7ws5r/perf-node-gather-daemonset-rgxc5"
Sep 10 17:52:29.059363 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:29.059255    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8dc495f6d-rt6q6_c4aa8ddd-f642-4ce0-b313-8684380b650f/manager/0.log"
Sep 10 17:52:29.076627 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:29.076598    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8dc495f6d-rt6q6_c4aa8ddd-f642-4ce0-b313-8684380b650f/copy-manifests/0.log"
Sep 10 17:52:30.526944 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:30.526907    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8655464fff-lvz59_69b6f0c6-12c7-45b7-92c0-9615b59c9afb/manager/0.log"
Sep 10 17:52:31.194354 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:31.194318    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz_4daf4486-5c21-4743-ba60-484fbfc18dd9/istio-proxy/0.log"
Sep 10 17:52:33.134561 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:33.134516    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799ccnjlz_4daf4486-5c21-4743-ba60-484fbfc18dd9/istio-proxy/0.log"
Sep 10 17:52:35.120979 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:35.120930    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mn5ts_47655ffd-db4a-4e92-a4f8-d170c8681d8c/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:37.899029 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:37.898997    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tp7gl_9efe55c6-4eef-4472-846d-4c3861daefa1/kube-multus-additional-cni-plugins/0.log"
Sep 10 17:52:37.933813 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:37.933777    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tp7gl_9efe55c6-4eef-4472-846d-4c3861daefa1/egress-router-binary-copy/0.log"
Sep 10 17:52:37.972396 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:37.972363    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tp7gl_9efe55c6-4eef-4472-846d-4c3861daefa1/cni-plugins/0.log"
Sep 10 17:52:38.025314 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:38.025281    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tp7gl_9efe55c6-4eef-4472-846d-4c3861daefa1/bond-cni-plugin/0.log"
Sep 10 17:52:38.059440 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:38.059400    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tp7gl_9efe55c6-4eef-4472-846d-4c3861daefa1/routeoverride-cni/0.log"
Sep 10 17:52:38.099413 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:38.099378    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tp7gl_9efe55c6-4eef-4472-846d-4c3861daefa1/whereabouts-cni-bincopy/0.log"
Sep 10 17:52:38.137330 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:38.137285    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tp7gl_9efe55c6-4eef-4472-846d-4c3861daefa1/whereabouts-cni/0.log"
Sep 10 17:52:38.223331 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:38.223244    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gmg9v_2fa83261-0f33-4437-b34b-ccb301ff0ac9/kube-multus/0.log"
Sep 10 17:52:38.391376 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:38.391347    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rzqqc_7a5b4316-2391-4e12-9a80-12ace78ee49d/network-metrics-daemon/0.log"
Sep 10 17:52:38.412326 ip-10-0-138-123 kubenswrapper[2566]: I0910 17:52:38.412274    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rzqqc_7a5b4316-2391-4e12-9a80-12ace78ee49d/kube-rbac-proxy/0.log"