Sep 18 18:37:08.752125 ip-10-0-130-118 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 18 18:37:08.752137 ip-10-0-130-118 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 18 18:37:08.752144 ip-10-0-130-118 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 18 18:37:08.752309 ip-10-0-130-118 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 18 18:37:18.934747 ip-10-0-130-118 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 18 18:37:18.934766 ip-10-0-130-118 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 414f45dd308f42b8bdfa9d7e03329ab0 --
Sep 18 18:39:22.167073 ip-10-0-130-118 systemd[1]: Starting Kubernetes Kubelet...
Sep 18 18:39:25.619280 ip-10-0-130-118 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 18:39:25.619280 ip-10-0-130-118 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 18 18:39:25.619280 ip-10-0-130-118 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 18:39:25.619280 ip-10-0-130-118 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 18 18:39:25.619280 ip-10-0-130-118 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 18:39:25.620136 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.619359    2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 18 18:39:25.624395 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624380    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:39:25.624395 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624395    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624399    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624402    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624406    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624409    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624412    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624415    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624418    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624421    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624424    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624426    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624430    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624433    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624435    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624438    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624440    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624443    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624445    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624449    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:39:25.624460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624451    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624454    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624457    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624460    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624463    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624466    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624469    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624472    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624475    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624477    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624480    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624484    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624488    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624492    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624496    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624499    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624502    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624506    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624509    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:39:25.624937 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624512    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624514    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624517    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624520    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624523    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624525    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624528    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624531    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624533    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624536    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624538    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624541    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624543    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624546    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624548    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624552    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624554    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624557    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624560    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624562    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:39:25.625468 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624565    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624568    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624570    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624573    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624575    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624578    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624580    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624583    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624585    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624588    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624590    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624593    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624596    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624598    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624601    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624603    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624607    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624610    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624612    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624615    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:39:25.625978 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624617    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624620    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624624    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624627    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624629    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624632    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624634    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624638    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624642    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624645    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624648    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624651    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624654    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624657    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624660    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624662    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624665    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624667    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624670    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.624672    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:39:25.626466 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625481    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625489    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625496    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625499    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625508    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625511    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625515    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625520    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625523    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625525    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625528    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625531    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625533    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625536    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625538    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625542    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625546    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625549    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625552    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625555    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:39:25.626961 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625558    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625560    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625563    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625565    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625568    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625570    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625573    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625575    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625578    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625581    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625583    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625586    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625588    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625591    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625594    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625597    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625599    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625608    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625610    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625613    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:39:25.627488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625615    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625619    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625622    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625625    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625628    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625631    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625634    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625637    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625640    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625642    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625645    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625648    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625650    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625653    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625655    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625657    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625660    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625668    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625671    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625673    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:39:25.628004 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625676    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625678    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625681    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625683    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625685    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625688    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625690    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625693    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625695    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625697    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625708    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625710    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625713    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625716    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625718    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625721    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625723    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625726    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625729    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:39:25.628501 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625732    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625735    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625738    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625741    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625743    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625746    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625748    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625751    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625754    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625756    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625759    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625761    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625764    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625767    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625769    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625771    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625774    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625776    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625779    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.625781    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:39:25.628990 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626223    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626228    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626231    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626239    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626242    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626244    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626247    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626249    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626252    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626255    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626258    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626261    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626264    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626266    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626269    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626272    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626274    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626277    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626279    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626282    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:39:25.629519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626285    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626287    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626290    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626292    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626295    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626297    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626302    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626305    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626308    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626311    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626314    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626316    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626319    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626322    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626325    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626327    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626336    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626339    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626341    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:39:25.630025 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626344    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626347    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626351    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626354    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626357    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626359    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626362    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626365    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626367    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626370    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626372    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626375    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626377    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626380    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626382    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626385    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626387    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626390    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626393    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:39:25.630485 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626396    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626398    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626401    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626403    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626406    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626408    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626411    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626414    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626417    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626419    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626422    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626430    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626433    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626435    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626437    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626441    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626444    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626447    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626450    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626454    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:39:25.630960 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626457    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626460    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626463    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626466    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626469    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626471    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626478    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626480    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626483    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626486    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626489    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626493    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626496    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626499    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626501    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626504    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626506    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626509    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626512    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:39:25.631450 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626515    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.626517    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626586    2558 flags.go:64] FLAG: --address="0.0.0.0"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626617    2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626626    2558 flags.go:64] FLAG: --anonymous-auth="true"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626639    2558 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626643    2558 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626646    2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626651    2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626655    2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626659    2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626662    2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626665    2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626668    2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626672    2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626675    2558 flags.go:64] FLAG: --cgroup-root=""
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626678    2558 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626681    2558 flags.go:64] FLAG: --client-ca-file=""
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626683    2558 flags.go:64] FLAG: --cloud-provider="external"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626686    2558 flags.go:64] FLAG: --cluster-dns="[]"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626693    2558 flags.go:64] FLAG: --cluster-domain=""
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626695    2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626698    2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626701    2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 18 18:39:25.631951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626706    2558 flags.go:64] FLAG: --container-log-max-files="5"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626710    2558 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626713    2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626716    2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626719    2558 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626722    2558 flags.go:64] FLAG: --contention-profiling="false"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626725    2558 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626728    2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626731    2558 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626734    2558 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626738    2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626741    2558 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626744    2558 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626747    2558 flags.go:64] FLAG: --enable-load-reader="false"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626762    2558 flags.go:64] FLAG: --enable-server="true"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626766    2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626774    2558 flags.go:64] FLAG: --event-burst="100"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626777    2558 flags.go:64] FLAG: --event-qps="50"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626780    2558 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626783    2558 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626786    2558 flags.go:64] FLAG: --eviction-hard=""
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626793    2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626796    2558 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626799    2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626803    2558 flags.go:64] FLAG: --eviction-soft=""
Sep 18 18:39:25.632536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626805    2558 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626808    2558 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626811    2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626814    2558 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626829    2558 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626832    2558 flags.go:64] FLAG: --fail-swap-on="true"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626835    2558 flags.go:64] FLAG: --feature-gates=""
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626841    2558 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626844    2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626847    2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626850    2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626854    2558 flags.go:64] FLAG: --healthz-port="10248"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626857    2558 flags.go:64] FLAG: --help="false"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626860    2558 flags.go:64] FLAG: --hostname-override="ip-10-0-130-118.ec2.internal"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626863    2558 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626866    2558 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626869    2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626872    2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626876    2558 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626879    2558 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626882    2558 flags.go:64] FLAG: --image-service-endpoint=""
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626885    2558 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626894    2558 flags.go:64] FLAG: --kube-api-burst="100"
Sep 18 18:39:25.633173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626897    2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626900    2558 flags.go:64] FLAG: --kube-api-qps="50"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626903    2558 flags.go:64] FLAG: --kube-reserved=""
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626906    2558 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626909    2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626912    2558 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626915    2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626918    2558 flags.go:64] FLAG: --lock-file=""
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626921    2558 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626924    2558 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626927    2558 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626932    2558 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626935    2558 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626938    2558 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626941    2558 flags.go:64] FLAG: --logging-format="text"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626943    2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626948    2558 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626951    2558 flags.go:64] FLAG: --manifest-url=""
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626954    2558 flags.go:64] FLAG: --manifest-url-header=""
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626958    2558 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626962    2558 flags.go:64] FLAG: --max-open-files="1000000"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626966    2558 flags.go:64] FLAG: --max-pods="110"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626969    2558 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626972    2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626975    2558 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 18 18:39:25.633971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626978    2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626981    2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626984    2558 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626987    2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626993    2558 flags.go:64] FLAG: --node-status-max-images="50"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626996    2558 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.626999    2558 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627008    2558 flags.go:64] FLAG: --pod-cidr=""
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627011    2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627015    2558 flags.go:64] FLAG: --pod-manifest-path=""
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627018    2558 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627021    2558 flags.go:64] FLAG: --pods-per-core="0"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627024    2558 flags.go:64] FLAG: --port="10250"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627027    2558 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627030    2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-005515162edb6fbad"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627033    2558 flags.go:64] FLAG: --qos-reserved=""
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627036    2558 flags.go:64] FLAG: --read-only-port="10255"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627039    2558 flags.go:64] FLAG: --register-node="true"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627042    2558 flags.go:64] FLAG: --register-with-taints=""
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627045    2558 flags.go:64] FLAG: --registry-burst="10"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627048    2558 flags.go:64] FLAG: --registry-qps="5"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627051    2558 flags.go:64] FLAG: --reserved-cpus=""
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627054    2558 flags.go:64] FLAG: --reserved-memory=""
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627060    2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627063    2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 18 18:39:25.634941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627066    2558 flags.go:64] FLAG: --rotate-certificates="false"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627069    2558 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627072    2558 flags.go:64] FLAG: --runonce="false"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627074    2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627077    2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627080    2558 flags.go:64] FLAG: --seccomp-default="false"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627083    2558 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627086    2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627089    2558 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627093    2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627098    2558 flags.go:64] FLAG: --storage-driver-password="root"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627101    2558 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627103    2558 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627106    2558 flags.go:64] FLAG: --storage-driver-user="root"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627109    2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627118    2558 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627121    2558 flags.go:64] FLAG: --system-cgroups=""
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627124    2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627129    2558 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627132    2558 flags.go:64] FLAG: --tls-cert-file=""
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627134    2558 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627140    2558 flags.go:64] FLAG: --tls-min-version=""
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627143    2558 flags.go:64] FLAG: --tls-private-key-file=""
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627146    2558 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627149    2558 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 18 18:39:25.635585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627152    2558 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627160    2558 flags.go:64] FLAG: --v="2"
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627164    2558 flags.go:64] FLAG: --version="false"
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627168    2558 flags.go:64] FLAG: --vmodule=""
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627172    2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.627176    2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627329    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627334    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627337    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627340    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627342    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627345    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627347    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627350    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627353    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627356    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627358    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627362    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627365    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627368    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627370    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:39:25.636236 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627373    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:39:25.636759 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627375    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:39:25.636759 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627385    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:39:25.636759 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627388    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:39:25.636759 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627390    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:39:25.636759 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627393    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:39:25.636759 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627397    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:39:25.636759 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627401    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:39:25.636759 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627406    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:39:25.636759 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627409    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:39:25.636759 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627412    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:39:25.636759 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627414    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:39:25.636759 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627417    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:39:25.636759 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627420    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:39:25.636759 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627422    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:39:25.636759 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627425    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:39:25.636759 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627429    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:39:25.636759 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627431    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:39:25.636759 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627434    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627436    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627439    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627442    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627444    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627447    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627450    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627452    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627455    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627458    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627461    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627464    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627467    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627470    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627472    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627475    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627477    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627485    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627488    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627491    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:39:25.637233 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627494    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627496    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627499    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627502    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627505    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627507    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627510    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627512    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627515    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627517    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627527    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627529    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627532    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627535    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627537    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627540    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627542    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627545    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627548    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627553    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:39:25.637732 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627555    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627558    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627561    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627564    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627567    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627570    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627572    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627575    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627578    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627580    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627588    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627591    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627593    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627596    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627599    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627601    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627604    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627606    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627609    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627611    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:39:25.638272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627613    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:39:25.638770 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627616    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:39:25.638770 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627620    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:39:25.638770 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627623    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:39:25.638770 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627627    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:39:25.638770 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.627630    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:39:25.638770 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.628135    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 18:39:25.638770 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.636449    2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 18 18:39:25.638770 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.636465    2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 18 18:39:25.638770 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.636485    2558 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 18 18:39:25.638770 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.636491    2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 18 18:39:25.638770 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636552    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:39:25.638770 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636557    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:39:25.638770 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636560    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:39:25.638770 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636563    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636566    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636569    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636572    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636574    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636577    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636580    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636583    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636585    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636588    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636591    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636595    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636598    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636601    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636604    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636607    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636610    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636613    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636616    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:39:25.639146 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636619    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636621    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636624    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636626    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636629    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636631    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636635    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636638    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636641    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636644    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636646    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636649    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636652    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636654    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636657    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636661    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636665    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636668    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636671    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636674    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:39:25.639653 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636677    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636680    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636682    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636685    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636688    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636691    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636694    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636696    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636699    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636701    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636704    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636706    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636709    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636712    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636714    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636717    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636719    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636721    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636724    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636728    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:39:25.640157 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636730    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636733    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636736    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636738    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636741    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636743    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636745    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636748    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636752    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636755    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636758    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636761    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636764    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636767    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636770    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636773    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636775    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636783    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636786    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:39:25.640641 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636788    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:39:25.641132 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636791    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:39:25.641132 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636793    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:39:25.641132 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636796    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:39:25.641132 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636798    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:39:25.641132 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636801    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:39:25.641132 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636804    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:39:25.641132 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636806    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:39:25.641132 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636809    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:39:25.641132 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636811    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:39:25.641132 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636814    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:39:25.641132 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636833    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:39:25.641132 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636838    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:39:25.641132 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636841    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:39:25.641132 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636844    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:39:25.641132 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636847    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:39:25.641132 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636850    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:39:25.641132 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636852    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:39:25.641559 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.636857    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 18:39:25.641559 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636980    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:39:25.641559 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636986    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:39:25.641559 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636989    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:39:25.641559 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636992    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:39:25.641559 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636995    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:39:25.641559 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.636998    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:39:25.641559 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637001    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:39:25.641559 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637003    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:39:25.641559 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637006    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:39:25.641559 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637009    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:39:25.641559 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637012    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:39:25.641559 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637014    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:39:25.641559 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637017    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:39:25.641559 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637020    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637023    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637025    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637027    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637030    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637032    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637035    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637037    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637040    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637042    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637045    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637048    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637051    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637054    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637057    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637059    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637062    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637065    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637068    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:39:25.642082 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637072    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637075    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637078    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637081    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637083    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637086    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637088    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637091    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637093    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637096    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637098    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637101    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637104    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637107    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637110    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637113    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637116    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637119    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637121    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637124    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:39:25.642543 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637126    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637129    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637131    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637134    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637137    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637140    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637143    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637145    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637148    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637150    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637152    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637155    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637159    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637162    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637165    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637168    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637171    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637174    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637177    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:39:25.643055 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637179    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637182    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637185    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637188    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637190    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637193    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637196    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637199    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637202    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637205    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637208    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637211    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637213    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637216    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637218    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637221    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637224    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637227    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637229    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637232    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:39:25.643533 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637235    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:39:25.644045 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637238    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:39:25.644045 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637241    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:39:25.644045 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637244    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:39:25.644045 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637246    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:39:25.644045 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637249    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:39:25.644045 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637252    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:39:25.644045 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:25.637254    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:39:25.644045 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.637259    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 18:39:25.644045 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.637385    2558 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 18 18:39:25.644045 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.639904    2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 18 18:39:25.644045 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.640704    2558 server.go:1019] "Starting client certificate rotation"
Sep 18 18:39:25.644045 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.640838    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 18:39:25.644045 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.640875    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 18:39:25.644417 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.643096    2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 18:39:25.644417 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.644294    2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 18:39:25.656676 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.656651    2558 log.go:25] "Validated CRI v1 runtime API"
Sep 18 18:39:25.659012 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.658996    2558 log.go:25] "Validated CRI v1 image API"
Sep 18 18:39:25.659869 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.659851    2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 18 18:39:25.662531 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.662498    2558 fs.go:135] Filesystem UUIDs: map[4fd7a961-533f-44b6-9631-3c8d0214fd82:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f6525907-0431-4aba-a1c0-b5d32bc22550:/dev/nvme0n1p4]
Sep 18 18:39:25.662713 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.662660    2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 18 18:39:25.667754 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.667645    2558 manager.go:216] Machine: {Timestamp:2026-09-18 18:39:25.66717951 +0000 UTC m=+1.865663727 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099831 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cb46cf8b1581db0bb0e48b8c73611 SystemUUID:ec2cb46c-f8b1-581d-b0bb-0e48b8c73611 BootID:414f45dd-308f-42b8-bdfa-9d7e03329ab0 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:df:da:66:25 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:df:da:66:25 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:fb:d6:d0:0a:18 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 18 18:39:25.667754 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.667744    2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 18 18:39:25.667920 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.667831    2558 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 18 18:39:25.669255 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.669223    2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 18 18:39:25.669423 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.669259    2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-118.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 18 18:39:25.670366 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.670354    2558 topology_manager.go:138] "Creating topology manager with none policy"
Sep 18 18:39:25.670430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.670372    2558 container_manager_linux.go:309] "Creating device plugin manager"
Sep 18 18:39:25.670430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.670384    2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 18:39:25.670430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.670399    2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 18:39:25.670571 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.670488    2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 18 18:39:25.673372 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.673357    2558 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 18:39:25.673514 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.673502    2558 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 18 18:39:25.673577 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.673539    2558 kubelet.go:486] "Attempting to sync node with API server"
Sep 18 18:39:25.673577 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.673556    2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 18 18:39:25.673577 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.673575    2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 18 18:39:25.673714 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.673593    2558 kubelet.go:398] "Adding apiserver pod source"
Sep 18 18:39:25.673714 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.673605    2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 18 18:39:25.674069 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.674055    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 18:39:25.674147 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.674081    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 18:39:25.674271 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.674255    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 18:39:25.676183 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.676162    2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 18 18:39:25.678353 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.678338    2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 18 18:39:25.678417 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.678368    2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 18 18:39:25.680308 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.680294    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 18 18:39:25.680362 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.680315    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 18 18:39:25.680362 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.680324    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 18 18:39:25.680362 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.680330    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 18 18:39:25.680362 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.680339    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 18 18:39:25.680362 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.680344    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 18 18:39:25.680362 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.680357    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 18 18:39:25.680362 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.680362    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 18 18:39:25.680574 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.680369    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 18 18:39:25.680574 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.680375    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 18 18:39:25.680574 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.680383    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 18 18:39:25.680574 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.680388    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 18 18:39:25.680574 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.680413    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 18 18:39:25.680574 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.680419    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 18 18:39:25.684094 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.684081    2558 server.go:1268] "Started kubelet"
Sep 18 18:39:25.684322 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.684290    2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 18 18:39:25.684481 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.684459    2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 18 18:39:25.684562 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.684509    2558 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 18 18:39:25.684715 ip-10-0-130-118 systemd[1]: Started Kubernetes Kubelet.
Sep 18 18:39:25.685394 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.685379    2558 server.go:310] "Adding debug handlers to kubelet server"
Sep 18 18:39:25.685583 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.685535    2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 18 18:39:25.688197 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:25.688178    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 18 18:39:25.688280 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:25.688238    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 18 18:39:25.688368 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.688347    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 18 18:39:25.688676 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:25.688659    2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 18 18:39:25.692450 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.692433    2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 18 18:39:25.692450 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.692448    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 18 18:39:25.692928 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:25.692906    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-118.ec2.internal\" not found"
Sep 18 18:39:25.692928 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.692926    2558 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 18 18:39:25.693057 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.692940    2558 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 18 18:39:25.693057 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.693036    2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 18 18:39:25.693183 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.693173    2558 reconstruct.go:97] "Volume reconstruction finished"
Sep 18 18:39:25.693183 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.693184    2558 reconciler.go:29] "Reconciler: start to sync state"
Sep 18 18:39:25.714704 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:25.714678    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 18 18:39:25.714795 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:25.714685    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 18 18:39:25.715373 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.715352    2558 factory.go:55] Registering systemd factory
Sep 18 18:39:25.715373 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.715365    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-spw9d"
Sep 18 18:39:25.715373 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.715377    2558 factory.go:223] Registration of the systemd container factory successfully
Sep 18 18:39:25.715557 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.715380    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-spw9d"
Sep 18 18:39:25.715728 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.715706    2558 factory.go:153] Registering CRI-O factory
Sep 18 18:39:25.715849 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.715757    2558 factory.go:223] Registration of the crio container factory successfully
Sep 18 18:39:25.715849 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.715808    2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 18 18:39:25.715955 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.715849    2558 factory.go:103] Registering Raw factory
Sep 18 18:39:25.715955 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.715870    2558 manager.go:1196] Started watching for new ooms in manager
Sep 18 18:39:25.716283 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.716262    2558 manager.go:349] Starting recovery of all containers
Sep 18 18:39:25.720967 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:25.720083    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-118.ec2.internal.18d67e4bbcf3399b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-118.ec2.internal,UID:ip-10-0-130-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-118.ec2.internal,},FirstTimestamp:2026-09-18 18:39:25.684054427 +0000 UTC m=+1.882538623,LastTimestamp:2026-09-18 18:39:25.684054427 +0000 UTC m=+1.882538623,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-118.ec2.internal,}"
Sep 18 18:39:25.726923 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.726908    2558 manager.go:354] Recovery completed
Sep 18 18:39:25.730879 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.730866    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:39:25.732838 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.732808    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:39:25.732903 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.732859    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:39:25.732903 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.732877    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:39:25.733456 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.733439    2558 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 18 18:39:25.733456 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.733453    2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 18 18:39:25.733586 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.733470    2558 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 18:39:25.733586 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.733473    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:39:25.734196 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.734173    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:39:25.734281 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.734208    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:39:25.734281 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.734220    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:39:25.734449 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.734435    2558 policy_none.go:49] "None policy: Start"
Sep 18 18:39:25.734505 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.734451    2558 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 18 18:39:25.734505 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.734461    2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 18 18:39:25.735365 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.735353    2558 policy_none.go:47] "Start"
Sep 18 18:39:25.767409 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.767394    2558 manager.go:337] "Starting Device Plugin manager"
Sep 18 18:39:25.774917 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:25.767438    2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 18 18:39:25.774917 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.767448    2558 server.go:87] "Starting device plugin registration server"
Sep 18 18:39:25.774917 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.767675    2558 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 18 18:39:25.774917 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.767692    2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 18 18:39:25.774917 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.767789    2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 18 18:39:25.774917 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.768364    2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 18 18:39:25.774917 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.768373    2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 18 18:39:25.774917 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:25.768834    2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 18 18:39:25.774917 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:25.768889    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-118.ec2.internal\" not found"
Sep 18 18:39:25.836727 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.836703    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 18 18:39:25.837931 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.837917    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 18 18:39:25.838016 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.837958    2558 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 18 18:39:25.838016 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.837979    2558 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 18 18:39:25.838016 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:25.838013    2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 18 18:39:25.839812 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.839789    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:39:25.868744 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.868720    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:39:25.869989 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.869942    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:39:25.869989 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.869974    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:39:25.869989 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.869989    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:39:25.870166 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.870021    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-118.ec2.internal"
Sep 18 18:39:25.876546 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.876531    2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-118.ec2.internal"
Sep 18 18:39:25.876630 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:25.876556    2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-118.ec2.internal\": node \"ip-10-0-130-118.ec2.internal\" not found"
Sep 18 18:39:25.891571 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:25.891550    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-118.ec2.internal\" not found"
Sep 18 18:39:25.938470 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.938442    2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-118.ec2.internal"]
Sep 18 18:39:25.938601 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.938514    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:39:25.940043 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.940029    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:39:25.940105 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.940056    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:39:25.940105 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.940073    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:39:25.941247 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.941234    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:39:25.941350 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.941335    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal"
Sep 18 18:39:25.941411 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.941364    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:39:25.942164 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.942143    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:39:25.942235 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.942169    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:39:25.942235 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.942171    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:39:25.942235 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.942185    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:39:25.942235 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.942190    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:39:25.942235 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.942200    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:39:25.943325 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.943312    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-118.ec2.internal"
Sep 18 18:39:25.943386 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.943335    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:39:25.943999 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.943984    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:39:25.944055 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.944016    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:39:25.944055 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.944026    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:39:25.959350 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:25.959331    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-118.ec2.internal\" not found" node="ip-10-0-130-118.ec2.internal"
Sep 18 18:39:25.963915 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:25.963901    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-118.ec2.internal\" not found" node="ip-10-0-130-118.ec2.internal"
Sep 18 18:39:25.992334 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:25.992318    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-118.ec2.internal\" not found"
Sep 18 18:39:25.995613 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.995588    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/396528f9074091ccc13f49651d4bd352-config\") pod \"kube-apiserver-proxy-ip-10-0-130-118.ec2.internal\" (UID: \"396528f9074091ccc13f49651d4bd352\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-118.ec2.internal"
Sep 18 18:39:25.995677 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.995621    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/db5792ee010cc74c32e3052501401cfb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal\" (UID: \"db5792ee010cc74c32e3052501401cfb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal"
Sep 18 18:39:25.995677 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:25.995639    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db5792ee010cc74c32e3052501401cfb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal\" (UID: \"db5792ee010cc74c32e3052501401cfb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal"
Sep 18 18:39:26.092613 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:26.092584    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-118.ec2.internal\" not found"
Sep 18 18:39:26.095872 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:26.095851    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/db5792ee010cc74c32e3052501401cfb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal\" (UID: \"db5792ee010cc74c32e3052501401cfb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal"
Sep 18 18:39:26.095931 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:26.095886    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db5792ee010cc74c32e3052501401cfb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal\" (UID: \"db5792ee010cc74c32e3052501401cfb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal"
Sep 18 18:39:26.095931 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:26.095922    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/396528f9074091ccc13f49651d4bd352-config\") pod \"kube-apiserver-proxy-ip-10-0-130-118.ec2.internal\" (UID: \"396528f9074091ccc13f49651d4bd352\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-118.ec2.internal"
Sep 18 18:39:26.096007 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:26.095927    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/db5792ee010cc74c32e3052501401cfb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal\" (UID: \"db5792ee010cc74c32e3052501401cfb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal"
Sep 18 18:39:26.096007 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:26.095965    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/396528f9074091ccc13f49651d4bd352-config\") pod \"kube-apiserver-proxy-ip-10-0-130-118.ec2.internal\" (UID: \"396528f9074091ccc13f49651d4bd352\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-118.ec2.internal"
Sep 18 18:39:26.096073 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:26.096015    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db5792ee010cc74c32e3052501401cfb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal\" (UID: \"db5792ee010cc74c32e3052501401cfb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal"
Sep 18 18:39:26.193371 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:26.193306    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-118.ec2.internal\" not found"
Sep 18 18:39:26.262645 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:26.262618    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal"
Sep 18 18:39:26.267591 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:26.267577    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-118.ec2.internal"
Sep 18 18:39:26.293695 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:26.293547    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-118.ec2.internal\" not found"
Sep 18 18:39:26.394067 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:26.394029    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-118.ec2.internal\" not found"
Sep 18 18:39:26.494499 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:26.494471    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-118.ec2.internal\" not found"
Sep 18 18:39:26.556657 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:26.556627    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:39:26.595395 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:26.595364    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-118.ec2.internal\" not found"
Sep 18 18:39:26.640676 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:26.640647    2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 18 18:39:26.641132 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:26.640791    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 18:39:26.641132 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:26.640845    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 18:39:26.695474 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:26.695452    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-118.ec2.internal\" not found"
Sep 18 18:39:26.719892 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:26.719856    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 18:34:25 +0000 UTC" deadline="2028-05-08 13:08:55.535756781 +0000 UTC"
Sep 18 18:39:26.719892 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:26.719887    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14346h29m28.815873284s"
Sep 18 18:39:26.796372 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:26.796345    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-118.ec2.internal\" not found"
Sep 18 18:39:26.807539 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:26.807510    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod396528f9074091ccc13f49651d4bd352.slice/crio-a4055c29334849a42098609325fb177ef20aec9218d316984ab7407146e7af4a WatchSource:0}: Error finding container a4055c29334849a42098609325fb177ef20aec9218d316984ab7407146e7af4a: Status 404 returned error can't find the container with id a4055c29334849a42098609325fb177ef20aec9218d316984ab7407146e7af4a
Sep 18 18:39:26.811978 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:26.811961    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:39:26.841073 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:26.841047    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb5792ee010cc74c32e3052501401cfb.slice/crio-d0858ebb795e61db7f111758407065e1ecd32f8c04e1c4f538f17d69468109d1 WatchSource:0}: Error finding container d0858ebb795e61db7f111758407065e1ecd32f8c04e1c4f538f17d69468109d1: Status 404 returned error can't find the container with id d0858ebb795e61db7f111758407065e1ecd32f8c04e1c4f538f17d69468109d1
Sep 18 18:39:26.842004 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:26.841954    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-118.ec2.internal" event={"ID":"396528f9074091ccc13f49651d4bd352","Type":"ContainerStarted","Data":"a4055c29334849a42098609325fb177ef20aec9218d316984ab7407146e7af4a"}
Sep 18 18:39:26.890049 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:26.890024    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:39:26.897235 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:26.897214    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-118.ec2.internal\" not found"
Sep 18 18:39:26.997711 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:26.997680    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-118.ec2.internal\" not found"
Sep 18 18:39:27.098201 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:27.098167    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-118.ec2.internal\" not found"
Sep 18 18:39:27.181417 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.181383    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:39:27.193240 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.193221    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal"
Sep 18 18:39:27.203777 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.203760    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 18:39:27.204590 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.204577    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-118.ec2.internal"
Sep 18 18:39:27.210625 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.210609    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 18:39:27.674460 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.674421    2558 apiserver.go:52] "Watching apiserver"
Sep 18 18:39:27.682102 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.682026    2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 18 18:39:27.682466 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.682442    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-zbxjj","openshift-network-operator/iptables-alerter-xwtjb","openshift-ovn-kubernetes/ovnkube-node-7bwsg","kube-system/global-pull-secret-syncer-2h6z8","kube-system/konnectivity-agent-8fj9b","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal","openshift-multus/multus-8mcxq","openshift-multus/multus-additional-cni-plugins-rb48x","openshift-multus/network-metrics-daemon-588ml","kube-system/kube-apiserver-proxy-ip-10-0-130-118.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg","openshift-cluster-node-tuning-operator/tuned-jq8qr","openshift-dns/node-resolver-sqcfr","openshift-image-registry/node-ca-qv5lr"]
Sep 18 18:39:27.684196 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.684108    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.686610 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.686587    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wbt22\""
Sep 18 18:39:27.686723 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.686697    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:39:27.686787 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.686764    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 18 18:39:27.689837 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.689799    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xwtjb"
Sep 18 18:39:27.691107 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.691072    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.692346 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.692182    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:27.692346 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.692233    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:39:27.692346 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:27.692255    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:39:27.692564 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.692545    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 18 18:39:27.692617 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.692585    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-frb9p\""
Sep 18 18:39:27.692617 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.692590    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 18 18:39:27.693013 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.692997    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 18 18:39:27.693569 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.693518    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8fj9b"
Sep 18 18:39:27.694058 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.693938    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 18 18:39:27.694058 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.694006    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 18 18:39:27.694058 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.694025    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 18 18:39:27.694276 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.694114    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kc2ts\""
Sep 18 18:39:27.694347 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.694333    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 18 18:39:27.694406 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.694374    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 18 18:39:27.694406 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.694388    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 18 18:39:27.694724 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.694705    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qv5lr"
Sep 18 18:39:27.695922 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.695901    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.698487 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.697159    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-trd8p\""
Sep 18 18:39:27.698487 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.697472    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 18 18:39:27.699037 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.698592    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-72zz2\""
Sep 18 18:39:27.699037 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.699004    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 18 18:39:27.699224 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.699203    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 18 18:39:27.699350 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.699333    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.700681 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.700003    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 18 18:39:27.700681 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.700019    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 18 18:39:27.700681 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.700282    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:27.700681 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:27.700370    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:39:27.702266 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.701799    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 18 18:39:27.702266 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.702019    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 18 18:39:27.702266 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.702156    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mb86q\""
Sep 18 18:39:27.702266 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.702203    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 18 18:39:27.702266 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.702235    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 18 18:39:27.702528 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.702317    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sqcfr"
Sep 18 18:39:27.702835 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.702800    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 18 18:39:27.702940 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.702806    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 18 18:39:27.702940 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.702876    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9qw7x\""
Sep 18 18:39:27.703189 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703168    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-etc-systemd\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.703269 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703199    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-lib-modules\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.703269 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703223    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-run-systemd\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.703269 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703246    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-cni-netd\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.703416 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703273    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.703416 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703297    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-run\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.703416 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703320    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1689135d-e785-446e-beb1-c106dd93ea55-tmp\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.703416 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703343    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-run-ovn\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.703416 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703367    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-log-socket\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.703416 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703391    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-etc-sysctl-d\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.703416 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703414    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af0b0fc3-baee-4f46-8ec0-83225a0cd5bf-host-slash\") pod \"iptables-alerter-xwtjb\" (UID: \"af0b0fc3-baee-4f46-8ec0-83225a0cd5bf\") " pod="openshift-network-operator/iptables-alerter-xwtjb"
Sep 18 18:39:27.703747 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703436    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vq46\" (UniqueName: \"kubernetes.io/projected/af0b0fc3-baee-4f46-8ec0-83225a0cd5bf-kube-api-access-6vq46\") pod \"iptables-alerter-xwtjb\" (UID: \"af0b0fc3-baee-4f46-8ec0-83225a0cd5bf\") " pod="openshift-network-operator/iptables-alerter-xwtjb"
Sep 18 18:39:27.703747 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703462    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-systemd-units\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.703747 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703470    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.703747 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703484    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-run-ovn-kubernetes\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.703747 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703508    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnd2l\" (UniqueName: \"kubernetes.io/projected/636f0979-fe2e-49fd-86d6-31bd7961e736-kube-api-access-bnd2l\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.703747 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703546    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3bd977ef-b9c9-4380-923a-57758db726b1-konnectivity-ca\") pod \"konnectivity-agent-8fj9b\" (UID: \"3bd977ef-b9c9-4380-923a-57758db726b1\") " pod="kube-system/konnectivity-agent-8fj9b"
Sep 18 18:39:27.703747 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703588    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-var-lib-kubelet\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.703747 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703619    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/af0b0fc3-baee-4f46-8ec0-83225a0cd5bf-iptables-alerter-script\") pod \"iptables-alerter-xwtjb\" (UID: \"af0b0fc3-baee-4f46-8ec0-83225a0cd5bf\") " pod="openshift-network-operator/iptables-alerter-xwtjb"
Sep 18 18:39:27.703747 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703643    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-kubelet\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.703747 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703676    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/636f0979-fe2e-49fd-86d6-31bd7961e736-env-overrides\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.703747 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703716    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-etc-modprobe-d\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.703747 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703743    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-etc-kubernetes\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.704430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703795    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv82w\" (UniqueName: \"kubernetes.io/projected/1689135d-e785-446e-beb1-c106dd93ea55-kube-api-access-cv82w\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.704430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703847    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-etc-openvswitch\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.704430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703881    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-run-openvswitch\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.704430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703930    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-node-log\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.704430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703960    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/636f0979-fe2e-49fd-86d6-31bd7961e736-ovnkube-script-lib\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.704430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.703989    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-etc-sysctl-conf\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.704430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.704012    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-sys\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.704430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.704032    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-host\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.704430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.704074    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-slash\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.704430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.704111    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-var-lib-openvswitch\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.704430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.704202    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/636f0979-fe2e-49fd-86d6-31bd7961e736-ovnkube-config\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.704430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.704230    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-etc-sysconfig\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.704430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.704262    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/636f0979-fe2e-49fd-86d6-31bd7961e736-ovn-node-metrics-cert\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.704430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.704287    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brpmq\" (UniqueName: \"kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq\") pod \"network-check-target-zbxjj\" (UID: \"f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237\") " pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:27.704430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.704310    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3bd977ef-b9c9-4380-923a-57758db726b1-agent-certs\") pod \"konnectivity-agent-8fj9b\" (UID: \"3bd977ef-b9c9-4380-923a-57758db726b1\") " pod="kube-system/konnectivity-agent-8fj9b"
Sep 18 18:39:27.704430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.704335    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1689135d-e785-446e-beb1-c106dd93ea55-etc-tuned\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.705101 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.704356    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-run-netns\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.705101 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.704378    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-cni-bin\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.705101 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.704833    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:27.705101 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:27.704892    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:39:27.705101 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.704922    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 18 18:39:27.705101 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.705011    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 18 18:39:27.705361 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.705124    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-w992t\""
Sep 18 18:39:27.705829 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.705800    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 18 18:39:27.706923 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.706226    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 18 18:39:27.706923 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.706415    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 18 18:39:27.706923 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.706506    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vzw98\""
Sep 18 18:39:27.707738 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.707717    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 18:39:27.725539 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.725518    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zmr2b"
Sep 18 18:39:27.736352 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.736332    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zmr2b"
Sep 18 18:39:27.793899 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.793877    2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 18 18:39:27.805230 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.805196    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-run-ovn\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.805356 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.805243    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-log-socket\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.805356 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.805286    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z7dx\" (UniqueName: \"kubernetes.io/projected/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-kube-api-access-4z7dx\") pod \"network-metrics-daemon-588ml\" (UID: \"ade7ca58-baa0-48f6-aed7-fe20bfe9461c\") " pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:27.805356 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.805315    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-host-run-netns\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.805356 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.805320    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-run-ovn\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.805594 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.805389    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-host-var-lib-cni-bin\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.805594 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.805420    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-host-var-lib-cni-multus\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.805594 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.805443    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/89672b2f-0680-4e2e-bdfa-66b6c41cacca-cnibin\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.805594 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.805445    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-log-socket\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.805594 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.805470    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-etc-sysctl-d\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.805890 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.805867    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vq46\" (UniqueName: \"kubernetes.io/projected/af0b0fc3-baee-4f46-8ec0-83225a0cd5bf-kube-api-access-6vq46\") pod \"iptables-alerter-xwtjb\" (UID: \"af0b0fc3-baee-4f46-8ec0-83225a0cd5bf\") " pod="openshift-network-operator/iptables-alerter-xwtjb"
Sep 18 18:39:27.805949 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.805895    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-etc-sysctl-d\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.805949 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.805917    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-run-ovn-kubernetes\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.806054 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.805959    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6b192a2e-3607-45c8-97aa-349c8b9503e0-kubelet-config\") pod \"global-pull-secret-syncer-2h6z8\" (UID: \"6b192a2e-3607-45c8-97aa-349c8b9503e0\") " pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:27.806054 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.805988    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-run-ovn-kubernetes\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.806054 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806038    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6b192a2e-3607-45c8-97aa-349c8b9503e0-dbus\") pod \"global-pull-secret-syncer-2h6z8\" (UID: \"6b192a2e-3607-45c8-97aa-349c8b9503e0\") " pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:27.806205 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806171    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-cni-binary-copy\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.806287 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806224    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-multus-daemon-config\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.806287 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806270    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/af0b0fc3-baee-4f46-8ec0-83225a0cd5bf-iptables-alerter-script\") pod \"iptables-alerter-xwtjb\" (UID: \"af0b0fc3-baee-4f46-8ec0-83225a0cd5bf\") " pod="openshift-network-operator/iptables-alerter-xwtjb"
Sep 18 18:39:27.806391 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806303    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-kubelet\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.806443 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806393    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/636f0979-fe2e-49fd-86d6-31bd7961e736-env-overrides\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.806494 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806469    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/281ef476-c8bd-4c7a-9357-5a7adb1f1ee7-hosts-file\") pod \"node-resolver-sqcfr\" (UID: \"281ef476-c8bd-4c7a-9357-5a7adb1f1ee7\") " pod="openshift-dns/node-resolver-sqcfr"
Sep 18 18:39:27.806559 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806544    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3ac8796c-5f58-4152-8474-88181c24214b-socket-dir\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.806559 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806544    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-kubelet\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.806621 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806591    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3ac8796c-5f58-4152-8474-88181c24214b-device-dir\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.806653 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806622    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-system-cni-dir\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.806653 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806645    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/89672b2f-0680-4e2e-bdfa-66b6c41cacca-cni-binary-copy\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.806715 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806689    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-etc-modprobe-d\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.806744 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806722    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv82w\" (UniqueName: \"kubernetes.io/projected/1689135d-e785-446e-beb1-c106dd93ea55-kube-api-access-cv82w\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.806796 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806783    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-etc-openvswitch\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.806852 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806815    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-run-openvswitch\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.806886 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806868    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-node-log\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.806920 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806897    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/636f0979-fe2e-49fd-86d6-31bd7961e736-ovnkube-script-lib\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.807008 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806983    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/636f0979-fe2e-49fd-86d6-31bd7961e736-env-overrides\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.807083 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807070    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-run-openvswitch\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.807113 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807072    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/af0b0fc3-baee-4f46-8ec0-83225a0cd5bf-iptables-alerter-script\") pod \"iptables-alerter-xwtjb\" (UID: \"af0b0fc3-baee-4f46-8ec0-83225a0cd5bf\") " pod="openshift-network-operator/iptables-alerter-xwtjb"
Sep 18 18:39:27.807147 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.806996    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqw7n\" (UniqueName: \"kubernetes.io/projected/89672b2f-0680-4e2e-bdfa-66b6c41cacca-kube-api-access-pqw7n\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.807147 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807119    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-node-log\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.807214 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807182    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-etc-sysctl-conf\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.807306 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807254    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-sys\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.807359 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807341    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-slash\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.807410 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807371    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/636f0979-fe2e-49fd-86d6-31bd7961e736-ovnkube-config\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.807410 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807373    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-etc-sysctl-conf\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.807539 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807445    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-etc-openvswitch\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.807539 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807484    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fcc6b30b-8557-41c1-93b4-50db142d69eb-serviceca\") pod \"node-ca-qv5lr\" (UID: \"fcc6b30b-8557-41c1-93b4-50db142d69eb\") " pod="openshift-image-registry/node-ca-qv5lr"
Sep 18 18:39:27.807539 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807511    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/281ef476-c8bd-4c7a-9357-5a7adb1f1ee7-tmp-dir\") pod \"node-resolver-sqcfr\" (UID: \"281ef476-c8bd-4c7a-9357-5a7adb1f1ee7\") " pod="openshift-dns/node-resolver-sqcfr"
Sep 18 18:39:27.807539 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807523    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-sys\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.807726 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807540    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-host-var-lib-kubelet\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.807726 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807574    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-etc-sysconfig\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.807726 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807639    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/636f0979-fe2e-49fd-86d6-31bd7961e736-ovn-node-metrics-cert\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.807726 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807671    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fcc6b30b-8557-41c1-93b4-50db142d69eb-host\") pod \"node-ca-qv5lr\" (UID: \"fcc6b30b-8557-41c1-93b4-50db142d69eb\") " pod="openshift-image-registry/node-ca-qv5lr"
Sep 18 18:39:27.807726 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807700    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtsbg\" (UniqueName: \"kubernetes.io/projected/281ef476-c8bd-4c7a-9357-5a7adb1f1ee7-kube-api-access-vtsbg\") pod \"node-resolver-sqcfr\" (UID: \"281ef476-c8bd-4c7a-9357-5a7adb1f1ee7\") " pod="openshift-dns/node-resolver-sqcfr"
Sep 18 18:39:27.807978 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807731    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-host-run-k8s-cni-cncf-io\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.807978 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807754    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-host-run-multus-certs\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.807978 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807844    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/636f0979-fe2e-49fd-86d6-31bd7961e736-ovnkube-script-lib\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.807978 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807863    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-run-netns\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.807978 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807895    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-cni-bin\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.807978 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807921    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3ac8796c-5f58-4152-8474-88181c24214b-registration-dir\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.807978 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807949    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-multus-conf-dir\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.807978 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807968    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-etc-modprobe-d\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.807978 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.807976    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-etc-systemd\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.808419 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.808004    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-lib-modules\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.808419 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.808035    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-cni-netd\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.808419 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.808043    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-etc-sysconfig\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.808419 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.808072    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/636f0979-fe2e-49fd-86d6-31bd7961e736-ovnkube-config\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.808419 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.808151    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-run-netns\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.808419 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.808363    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-cni-netd\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.808751 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.808531    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-etc-systemd\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.808751 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.808577    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-cni-bin\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.808751 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.808613    2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 18 18:39:27.808751 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.808645    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-lib-modules\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.809938 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.809893    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-slash\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.810035 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.809953    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-etc-kubernetes\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.810035 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.809982    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/89672b2f-0680-4e2e-bdfa-66b6c41cacca-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.810160 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810038    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1689135d-e785-446e-beb1-c106dd93ea55-tmp\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.810160 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810076    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f2t5\" (UniqueName: \"kubernetes.io/projected/fcc6b30b-8557-41c1-93b4-50db142d69eb-kube-api-access-8f2t5\") pod \"node-ca-qv5lr\" (UID: \"fcc6b30b-8557-41c1-93b4-50db142d69eb\") " pod="openshift-image-registry/node-ca-qv5lr"
Sep 18 18:39:27.810160 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810108    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret\") pod \"global-pull-secret-syncer-2h6z8\" (UID: \"6b192a2e-3607-45c8-97aa-349c8b9503e0\") " pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:27.810308 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810157    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-cnibin\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.810308 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810198    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86b47\" (UniqueName: \"kubernetes.io/projected/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-kube-api-access-86b47\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.810308 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810219    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/89672b2f-0680-4e2e-bdfa-66b6c41cacca-system-cni-dir\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.810308 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810280    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/89672b2f-0680-4e2e-bdfa-66b6c41cacca-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.810496 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810338    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af0b0fc3-baee-4f46-8ec0-83225a0cd5bf-host-slash\") pod \"iptables-alerter-xwtjb\" (UID: \"af0b0fc3-baee-4f46-8ec0-83225a0cd5bf\") " pod="openshift-network-operator/iptables-alerter-xwtjb"
Sep 18 18:39:27.810496 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810418    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-systemd-units\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.810597 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810529    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-systemd-units\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.810597 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810531    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnd2l\" (UniqueName: \"kubernetes.io/projected/636f0979-fe2e-49fd-86d6-31bd7961e736-kube-api-access-bnd2l\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.810702 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810628    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3bd977ef-b9c9-4380-923a-57758db726b1-konnectivity-ca\") pod \"konnectivity-agent-8fj9b\" (UID: \"3bd977ef-b9c9-4380-923a-57758db726b1\") " pod="kube-system/konnectivity-agent-8fj9b"
Sep 18 18:39:27.810702 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810666    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-var-lib-kubelet\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.810795 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810669    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af0b0fc3-baee-4f46-8ec0-83225a0cd5bf-host-slash\") pod \"iptables-alerter-xwtjb\" (UID: \"af0b0fc3-baee-4f46-8ec0-83225a0cd5bf\") " pod="openshift-network-operator/iptables-alerter-xwtjb"
Sep 18 18:39:27.810795 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810771    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-var-lib-kubelet\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.810917 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810871    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-etc-kubernetes\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.810917 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810890    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-etc-kubernetes\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.811016 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810942    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3ac8796c-5f58-4152-8474-88181c24214b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.811070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.810993    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3ac8796c-5f58-4152-8474-88181c24214b-etc-selinux\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.811124 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.811088    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3ac8796c-5f58-4152-8474-88181c24214b-sys-fs\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.811169 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.811153    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-host\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.811281 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.811261    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3bd977ef-b9c9-4380-923a-57758db726b1-konnectivity-ca\") pod \"konnectivity-agent-8fj9b\" (UID: \"3bd977ef-b9c9-4380-923a-57758db726b1\") " pod="kube-system/konnectivity-agent-8fj9b"
Sep 18 18:39:27.811373 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.811262    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-var-lib-openvswitch\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.811373 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.811270    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-host\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.812339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.811689    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-var-lib-openvswitch\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.812339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.811725    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3bd977ef-b9c9-4380-923a-57758db726b1-agent-certs\") pod \"konnectivity-agent-8fj9b\" (UID: \"3bd977ef-b9c9-4380-923a-57758db726b1\") " pod="kube-system/konnectivity-agent-8fj9b"
Sep 18 18:39:27.812339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.811762    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs\") pod \"network-metrics-daemon-588ml\" (UID: \"ade7ca58-baa0-48f6-aed7-fe20bfe9461c\") " pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:27.812339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.811791    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brpmq\" (UniqueName: \"kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq\") pod \"network-check-target-zbxjj\" (UID: \"f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237\") " pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:27.812339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.811829    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-multus-cni-dir\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.812339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.811851    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-hostroot\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.812339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.811871    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/89672b2f-0680-4e2e-bdfa-66b6c41cacca-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.812339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.811898    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1689135d-e785-446e-beb1-c106dd93ea55-etc-tuned\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.812339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.811919    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-multus-socket-dir-parent\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.812339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.811942    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-run-systemd\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.812339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.811962    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.812339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.811984    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74m7b\" (UniqueName: \"kubernetes.io/projected/3ac8796c-5f58-4152-8474-88181c24214b-kube-api-access-74m7b\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.812339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.812008    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-run\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.812339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.812137    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1689135d-e785-446e-beb1-c106dd93ea55-run\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.812339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.812193    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-run-systemd\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.812339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.812247    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/636f0979-fe2e-49fd-86d6-31bd7961e736-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.813365 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.812580    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/636f0979-fe2e-49fd-86d6-31bd7961e736-ovn-node-metrics-cert\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.813365 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.812959    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1689135d-e785-446e-beb1-c106dd93ea55-tmp\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.814448 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.814422    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1689135d-e785-446e-beb1-c106dd93ea55-etc-tuned\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.814927 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.814904    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv82w\" (UniqueName: \"kubernetes.io/projected/1689135d-e785-446e-beb1-c106dd93ea55-kube-api-access-cv82w\") pod \"tuned-jq8qr\" (UID: \"1689135d-e785-446e-beb1-c106dd93ea55\") " pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:27.815043 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.814913    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3bd977ef-b9c9-4380-923a-57758db726b1-agent-certs\") pod \"konnectivity-agent-8fj9b\" (UID: \"3bd977ef-b9c9-4380-923a-57758db726b1\") " pod="kube-system/konnectivity-agent-8fj9b"
Sep 18 18:39:27.815603 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.815527    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vq46\" (UniqueName: \"kubernetes.io/projected/af0b0fc3-baee-4f46-8ec0-83225a0cd5bf-kube-api-access-6vq46\") pod \"iptables-alerter-xwtjb\" (UID: \"af0b0fc3-baee-4f46-8ec0-83225a0cd5bf\") " pod="openshift-network-operator/iptables-alerter-xwtjb"
Sep 18 18:39:27.817627 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.817609    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnd2l\" (UniqueName: \"kubernetes.io/projected/636f0979-fe2e-49fd-86d6-31bd7961e736-kube-api-access-bnd2l\") pod \"ovnkube-node-7bwsg\" (UID: \"636f0979-fe2e-49fd-86d6-31bd7961e736\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:27.817885 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:27.817868    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:39:27.817965 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:27.817890    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:39:27.817965 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:27.817903    2558 projected.go:196] Error preparing data for projected volume kube-api-access-brpmq for pod openshift-network-diagnostics/network-check-target-zbxjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:27.818048 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:27.817977    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq podName:f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:28.317957045 +0000 UTC m=+4.516441240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-brpmq" (UniqueName: "kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq") pod "network-check-target-zbxjj" (UID: "f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:27.844107 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.844071    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal" event={"ID":"db5792ee010cc74c32e3052501401cfb","Type":"ContainerStarted","Data":"d0858ebb795e61db7f111758407065e1ecd32f8c04e1c4f538f17d69468109d1"}
Sep 18 18:39:27.912729 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.912701    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3ac8796c-5f58-4152-8474-88181c24214b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.912861 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.912734    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3ac8796c-5f58-4152-8474-88181c24214b-etc-selinux\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.912861 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.912756    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3ac8796c-5f58-4152-8474-88181c24214b-sys-fs\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.912861 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.912835    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs\") pod \"network-metrics-daemon-588ml\" (UID: \"ade7ca58-baa0-48f6-aed7-fe20bfe9461c\") " pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:27.912861 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.912851    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3ac8796c-5f58-4152-8474-88181c24214b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.913070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.912838    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3ac8796c-5f58-4152-8474-88181c24214b-sys-fs\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.913070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.912882    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-multus-cni-dir\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.913070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.912897    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3ac8796c-5f58-4152-8474-88181c24214b-etc-selinux\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.913070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.912909    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-hostroot\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.913070 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:27.912915    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:27.913070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.912942    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/89672b2f-0680-4e2e-bdfa-66b6c41cacca-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.913070 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:27.912977    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs podName:ade7ca58-baa0-48f6-aed7-fe20bfe9461c nodeName:}" failed. No retries permitted until 2026-09-18 18:39:28.412958734 +0000 UTC m=+4.611442930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs") pod "network-metrics-daemon-588ml" (UID: "ade7ca58-baa0-48f6-aed7-fe20bfe9461c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:27.913070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913008    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-multus-cni-dir\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.913070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.912972    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-hostroot\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.913070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.912996    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-multus-socket-dir-parent\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.913070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913059    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74m7b\" (UniqueName: \"kubernetes.io/projected/3ac8796c-5f58-4152-8474-88181c24214b-kube-api-access-74m7b\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.913567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913061    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-multus-socket-dir-parent\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.913567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913087    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z7dx\" (UniqueName: \"kubernetes.io/projected/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-kube-api-access-4z7dx\") pod \"network-metrics-daemon-588ml\" (UID: \"ade7ca58-baa0-48f6-aed7-fe20bfe9461c\") " pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:27.913567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913104    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/89672b2f-0680-4e2e-bdfa-66b6c41cacca-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.913567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913118    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-host-run-netns\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.913567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913143    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-host-var-lib-cni-bin\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.913567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913186    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-host-var-lib-cni-multus\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.913567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913209    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/89672b2f-0680-4e2e-bdfa-66b6c41cacca-cnibin\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.913567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913207    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-host-run-netns\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.913567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913242    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6b192a2e-3607-45c8-97aa-349c8b9503e0-kubelet-config\") pod \"global-pull-secret-syncer-2h6z8\" (UID: \"6b192a2e-3607-45c8-97aa-349c8b9503e0\") " pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:27.913567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913247    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/89672b2f-0680-4e2e-bdfa-66b6c41cacca-cnibin\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.913567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913249    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-host-var-lib-cni-multus\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.913567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913234    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-host-var-lib-cni-bin\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.913567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913303    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6b192a2e-3607-45c8-97aa-349c8b9503e0-kubelet-config\") pod \"global-pull-secret-syncer-2h6z8\" (UID: \"6b192a2e-3607-45c8-97aa-349c8b9503e0\") " pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:27.913567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913298    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6b192a2e-3607-45c8-97aa-349c8b9503e0-dbus\") pod \"global-pull-secret-syncer-2h6z8\" (UID: \"6b192a2e-3607-45c8-97aa-349c8b9503e0\") " pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:27.913567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913343    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-cni-binary-copy\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.913567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913355    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6b192a2e-3607-45c8-97aa-349c8b9503e0-dbus\") pod \"global-pull-secret-syncer-2h6z8\" (UID: \"6b192a2e-3607-45c8-97aa-349c8b9503e0\") " pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:27.913567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913375    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-multus-daemon-config\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.913567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913407    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/281ef476-c8bd-4c7a-9357-5a7adb1f1ee7-hosts-file\") pod \"node-resolver-sqcfr\" (UID: \"281ef476-c8bd-4c7a-9357-5a7adb1f1ee7\") " pod="openshift-dns/node-resolver-sqcfr"
Sep 18 18:39:27.914418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913457    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3ac8796c-5f58-4152-8474-88181c24214b-socket-dir\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.914418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913501    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3ac8796c-5f58-4152-8474-88181c24214b-device-dir\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.914418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913530    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/281ef476-c8bd-4c7a-9357-5a7adb1f1ee7-hosts-file\") pod \"node-resolver-sqcfr\" (UID: \"281ef476-c8bd-4c7a-9357-5a7adb1f1ee7\") " pod="openshift-dns/node-resolver-sqcfr"
Sep 18 18:39:27.914418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913532    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-system-cni-dir\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.914418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913580    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-system-cni-dir\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.914418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913593    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/89672b2f-0680-4e2e-bdfa-66b6c41cacca-cni-binary-copy\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.914418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913622    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3ac8796c-5f58-4152-8474-88181c24214b-socket-dir\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.914418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913634    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3ac8796c-5f58-4152-8474-88181c24214b-device-dir\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.914418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913657    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqw7n\" (UniqueName: \"kubernetes.io/projected/89672b2f-0680-4e2e-bdfa-66b6c41cacca-kube-api-access-pqw7n\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.914418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913690    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fcc6b30b-8557-41c1-93b4-50db142d69eb-serviceca\") pod \"node-ca-qv5lr\" (UID: \"fcc6b30b-8557-41c1-93b4-50db142d69eb\") " pod="openshift-image-registry/node-ca-qv5lr"
Sep 18 18:39:27.914418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913725    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/281ef476-c8bd-4c7a-9357-5a7adb1f1ee7-tmp-dir\") pod \"node-resolver-sqcfr\" (UID: \"281ef476-c8bd-4c7a-9357-5a7adb1f1ee7\") " pod="openshift-dns/node-resolver-sqcfr"
Sep 18 18:39:27.914418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913768    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-host-var-lib-kubelet\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.914418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913850    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fcc6b30b-8557-41c1-93b4-50db142d69eb-host\") pod \"node-ca-qv5lr\" (UID: \"fcc6b30b-8557-41c1-93b4-50db142d69eb\") " pod="openshift-image-registry/node-ca-qv5lr"
Sep 18 18:39:27.914418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913876    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtsbg\" (UniqueName: \"kubernetes.io/projected/281ef476-c8bd-4c7a-9357-5a7adb1f1ee7-kube-api-access-vtsbg\") pod \"node-resolver-sqcfr\" (UID: \"281ef476-c8bd-4c7a-9357-5a7adb1f1ee7\") " pod="openshift-dns/node-resolver-sqcfr"
Sep 18 18:39:27.914418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913900    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-host-run-k8s-cni-cncf-io\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.914418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913926    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-host-run-multus-certs\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.914418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913962    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-cni-binary-copy\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.914418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913963    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fcc6b30b-8557-41c1-93b4-50db142d69eb-host\") pod \"node-ca-qv5lr\" (UID: \"fcc6b30b-8557-41c1-93b4-50db142d69eb\") " pod="openshift-image-registry/node-ca-qv5lr"
Sep 18 18:39:27.915130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.913973    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3ac8796c-5f58-4152-8474-88181c24214b-registration-dir\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.915130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914000    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-multus-daemon-config\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.915130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914016    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-host-var-lib-kubelet\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.915130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914028    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3ac8796c-5f58-4152-8474-88181c24214b-registration-dir\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.915130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914014    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-multus-conf-dir\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.915130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914065    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-multus-conf-dir\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.915130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914072    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-host-run-multus-certs\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.915130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914060    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-host-run-k8s-cni-cncf-io\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.915130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914124    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/281ef476-c8bd-4c7a-9357-5a7adb1f1ee7-tmp-dir\") pod \"node-resolver-sqcfr\" (UID: \"281ef476-c8bd-4c7a-9357-5a7adb1f1ee7\") " pod="openshift-dns/node-resolver-sqcfr"
Sep 18 18:39:27.915130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914202    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fcc6b30b-8557-41c1-93b4-50db142d69eb-serviceca\") pod \"node-ca-qv5lr\" (UID: \"fcc6b30b-8557-41c1-93b4-50db142d69eb\") " pod="openshift-image-registry/node-ca-qv5lr"
Sep 18 18:39:27.915130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914207    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/89672b2f-0680-4e2e-bdfa-66b6c41cacca-cni-binary-copy\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.915130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914215    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-etc-kubernetes\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.915130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914266    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/89672b2f-0680-4e2e-bdfa-66b6c41cacca-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.915130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914303    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f2t5\" (UniqueName: \"kubernetes.io/projected/fcc6b30b-8557-41c1-93b4-50db142d69eb-kube-api-access-8f2t5\") pod \"node-ca-qv5lr\" (UID: \"fcc6b30b-8557-41c1-93b4-50db142d69eb\") " pod="openshift-image-registry/node-ca-qv5lr"
Sep 18 18:39:27.915130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914328    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret\") pod \"global-pull-secret-syncer-2h6z8\" (UID: \"6b192a2e-3607-45c8-97aa-349c8b9503e0\") " pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:27.915130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914334    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-etc-kubernetes\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.915130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914353    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-cnibin\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.915130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914376    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86b47\" (UniqueName: \"kubernetes.io/projected/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-kube-api-access-86b47\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.915665 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914418    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/89672b2f-0680-4e2e-bdfa-66b6c41cacca-system-cni-dir\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.915665 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914450    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-cnibin\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.915665 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914444    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/89672b2f-0680-4e2e-bdfa-66b6c41cacca-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.915665 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914512    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/89672b2f-0680-4e2e-bdfa-66b6c41cacca-system-cni-dir\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.915665 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:27.914522    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:27.915665 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:27.914571    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret podName:6b192a2e-3607-45c8-97aa-349c8b9503e0 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:28.414556316 +0000 UTC m=+4.613040502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret") pod "global-pull-secret-syncer-2h6z8" (UID: "6b192a2e-3607-45c8-97aa-349c8b9503e0") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:27.915665 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914701    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/89672b2f-0680-4e2e-bdfa-66b6c41cacca-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.915665 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.914869    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/89672b2f-0680-4e2e-bdfa-66b6c41cacca-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:27.921927 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.921900    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74m7b\" (UniqueName: \"kubernetes.io/projected/3ac8796c-5f58-4152-8474-88181c24214b-kube-api-access-74m7b\") pod \"aws-ebs-csi-driver-node-4jthg\" (UID: \"3ac8796c-5f58-4152-8474-88181c24214b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:27.922639 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.922521    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z7dx\" (UniqueName: \"kubernetes.io/projected/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-kube-api-access-4z7dx\") pod \"network-metrics-daemon-588ml\" (UID: \"ade7ca58-baa0-48f6-aed7-fe20bfe9461c\") " pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:27.922639 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.922627    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtsbg\" (UniqueName: \"kubernetes.io/projected/281ef476-c8bd-4c7a-9357-5a7adb1f1ee7-kube-api-access-vtsbg\") pod \"node-resolver-sqcfr\" (UID: \"281ef476-c8bd-4c7a-9357-5a7adb1f1ee7\") " pod="openshift-dns/node-resolver-sqcfr"
Sep 18 18:39:27.923005 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.922972    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f2t5\" (UniqueName: \"kubernetes.io/projected/fcc6b30b-8557-41c1-93b4-50db142d69eb-kube-api-access-8f2t5\") pod \"node-ca-qv5lr\" (UID: \"fcc6b30b-8557-41c1-93b4-50db142d69eb\") " pod="openshift-image-registry/node-ca-qv5lr"
Sep 18 18:39:27.923195 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.923170    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86b47\" (UniqueName: \"kubernetes.io/projected/8fe8cdb9-6f30-4e99-8f6d-9610f622eb52-kube-api-access-86b47\") pod \"multus-8mcxq\" (UID: \"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52\") " pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:27.923303 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:27.923287    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqw7n\" (UniqueName: \"kubernetes.io/projected/89672b2f-0680-4e2e-bdfa-66b6c41cacca-kube-api-access-pqw7n\") pod \"multus-additional-cni-plugins-rb48x\" (UID: \"89672b2f-0680-4e2e-bdfa-66b6c41cacca\") " pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:28.002370 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.002340    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jq8qr"
Sep 18 18:39:28.010633 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.010613    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xwtjb"
Sep 18 18:39:28.019941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.019922    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:28.026472 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.026456    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8fj9b"
Sep 18 18:39:28.032926 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.032908    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qv5lr"
Sep 18 18:39:28.040469 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.040452    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8mcxq"
Sep 18 18:39:28.049093 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.049074    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rb48x"
Sep 18 18:39:28.056557 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.056542    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sqcfr"
Sep 18 18:39:28.062048 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.062032    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg"
Sep 18 18:39:28.174389 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.174356    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:39:28.417424 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.417344    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs\") pod \"network-metrics-daemon-588ml\" (UID: \"ade7ca58-baa0-48f6-aed7-fe20bfe9461c\") " pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:28.417424 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.417387    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brpmq\" (UniqueName: \"kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq\") pod \"network-check-target-zbxjj\" (UID: \"f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237\") " pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:28.417620 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.417471    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret\") pod \"global-pull-secret-syncer-2h6z8\" (UID: \"6b192a2e-3607-45c8-97aa-349c8b9503e0\") " pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:28.417620 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:28.417514    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:28.417620 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:28.417572    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:28.417620 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:28.417585    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:39:28.417620 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:28.417605    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:39:28.417620 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:28.417618    2558 projected.go:196] Error preparing data for projected volume kube-api-access-brpmq for pod openshift-network-diagnostics/network-check-target-zbxjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:28.417881 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:28.417591    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs podName:ade7ca58-baa0-48f6-aed7-fe20bfe9461c nodeName:}" failed. No retries permitted until 2026-09-18 18:39:29.417574041 +0000 UTC m=+5.616058243 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs") pod "network-metrics-daemon-588ml" (UID: "ade7ca58-baa0-48f6-aed7-fe20bfe9461c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:28.417881 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:28.417669    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret podName:6b192a2e-3607-45c8-97aa-349c8b9503e0 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:29.417657518 +0000 UTC m=+5.616141703 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret") pod "global-pull-secret-syncer-2h6z8" (UID: "6b192a2e-3607-45c8-97aa-349c8b9503e0") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:28.417881 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:28.417686    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq podName:f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:29.417677592 +0000 UTC m=+5.616161786 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-brpmq" (UniqueName: "kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq") pod "network-check-target-zbxjj" (UID: "f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:28.602856 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.602808    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:39:28.700685 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:28.700649    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcc6b30b_8557_41c1_93b4_50db142d69eb.slice/crio-273f9cbd074b0f05ed8169267e9a4644b905be44b10f0b6888c49893d13e1ac8 WatchSource:0}: Error finding container 273f9cbd074b0f05ed8169267e9a4644b905be44b10f0b6888c49893d13e1ac8: Status 404 returned error can't find the container with id 273f9cbd074b0f05ed8169267e9a4644b905be44b10f0b6888c49893d13e1ac8
Sep 18 18:39:28.702659 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:28.702633    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ac8796c_5f58_4152_8474_88181c24214b.slice/crio-5cb4dbb43902864b3868298c49ddb3195bfa140d2472d3bd3b7127b208842cfe WatchSource:0}: Error finding container 5cb4dbb43902864b3868298c49ddb3195bfa140d2472d3bd3b7127b208842cfe: Status 404 returned error can't find the container with id 5cb4dbb43902864b3868298c49ddb3195bfa140d2472d3bd3b7127b208842cfe
Sep 18 18:39:28.707205 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:28.707182    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod636f0979_fe2e_49fd_86d6_31bd7961e736.slice/crio-b9c6086e5ea13f824d5e32fd0a7a3050375efde89d8b958513a31df46fbded30 WatchSource:0}: Error finding container b9c6086e5ea13f824d5e32fd0a7a3050375efde89d8b958513a31df46fbded30: Status 404 returned error can't find the container with id b9c6086e5ea13f824d5e32fd0a7a3050375efde89d8b958513a31df46fbded30
Sep 18 18:39:28.707763 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:28.707738    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fe8cdb9_6f30_4e99_8f6d_9610f622eb52.slice/crio-88c4f8af749d3b1dd91c385b770e013a883a2ebbb4acaf4f0b612a00839d47c7 WatchSource:0}: Error finding container 88c4f8af749d3b1dd91c385b770e013a883a2ebbb4acaf4f0b612a00839d47c7: Status 404 returned error can't find the container with id 88c4f8af749d3b1dd91c385b770e013a883a2ebbb4acaf4f0b612a00839d47c7
Sep 18 18:39:28.708700 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:28.708621    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bd977ef_b9c9_4380_923a_57758db726b1.slice/crio-69e38ca0a40ac57c389ca1ecbfcd375653a8bacf1148b5359919bd8329cf24f5 WatchSource:0}: Error finding container 69e38ca0a40ac57c389ca1ecbfcd375653a8bacf1148b5359919bd8329cf24f5: Status 404 returned error can't find the container with id 69e38ca0a40ac57c389ca1ecbfcd375653a8bacf1148b5359919bd8329cf24f5
Sep 18 18:39:28.710408 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:28.710313    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf0b0fc3_baee_4f46_8ec0_83225a0cd5bf.slice/crio-150d0c9b19857c7cc0db2bdbed352d1a1533a7eb03aa9503a7e280ce982ed4f4 WatchSource:0}: Error finding container 150d0c9b19857c7cc0db2bdbed352d1a1533a7eb03aa9503a7e280ce982ed4f4: Status 404 returned error can't find the container with id 150d0c9b19857c7cc0db2bdbed352d1a1533a7eb03aa9503a7e280ce982ed4f4
Sep 18 18:39:28.711542 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:28.711288    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod281ef476_c8bd_4c7a_9357_5a7adb1f1ee7.slice/crio-1c38ddd8339f8ff64f8d48a429b36c2a79d1d0c12dbaf6a8c60feee879d6b1f9 WatchSource:0}: Error finding container 1c38ddd8339f8ff64f8d48a429b36c2a79d1d0c12dbaf6a8c60feee879d6b1f9: Status 404 returned error can't find the container with id 1c38ddd8339f8ff64f8d48a429b36c2a79d1d0c12dbaf6a8c60feee879d6b1f9
Sep 18 18:39:28.712412 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:28.712353    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1689135d_e785_446e_beb1_c106dd93ea55.slice/crio-415b06c5d6ac5bc4ab1c0d0bf69144de4e2312b4fed80f36788a9f38be86fbfa WatchSource:0}: Error finding container 415b06c5d6ac5bc4ab1c0d0bf69144de4e2312b4fed80f36788a9f38be86fbfa: Status 404 returned error can't find the container with id 415b06c5d6ac5bc4ab1c0d0bf69144de4e2312b4fed80f36788a9f38be86fbfa
Sep 18 18:39:28.713327 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:39:28.713243    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89672b2f_0680_4e2e_bdfa_66b6c41cacca.slice/crio-ee733601d416ee7c2ff5a26fe3bfe0398a3eb29c2787d5f5f278bd990e8335dc WatchSource:0}: Error finding container ee733601d416ee7c2ff5a26fe3bfe0398a3eb29c2787d5f5f278bd990e8335dc: Status 404 returned error can't find the container with id ee733601d416ee7c2ff5a26fe3bfe0398a3eb29c2787d5f5f278bd990e8335dc
Sep 18 18:39:28.738886 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.738840    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 18:34:27 +0000 UTC" deadline="2028-05-27 09:40:00.329397756 +0000 UTC"
Sep 18 18:39:28.738886 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.738885    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14799h0m31.590515598s"
Sep 18 18:39:28.847374 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.847345    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jq8qr" event={"ID":"1689135d-e785-446e-beb1-c106dd93ea55","Type":"ContainerStarted","Data":"415b06c5d6ac5bc4ab1c0d0bf69144de4e2312b4fed80f36788a9f38be86fbfa"}
Sep 18 18:39:28.848520 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.848496    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sqcfr" event={"ID":"281ef476-c8bd-4c7a-9357-5a7adb1f1ee7","Type":"ContainerStarted","Data":"1c38ddd8339f8ff64f8d48a429b36c2a79d1d0c12dbaf6a8c60feee879d6b1f9"}
Sep 18 18:39:28.849758 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.849732    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8fj9b" event={"ID":"3bd977ef-b9c9-4380-923a-57758db726b1","Type":"ContainerStarted","Data":"69e38ca0a40ac57c389ca1ecbfcd375653a8bacf1148b5359919bd8329cf24f5"}
Sep 18 18:39:28.850971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.850930    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg" event={"ID":"636f0979-fe2e-49fd-86d6-31bd7961e736","Type":"ContainerStarted","Data":"b9c6086e5ea13f824d5e32fd0a7a3050375efde89d8b958513a31df46fbded30"}
Sep 18 18:39:28.853311 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.853291    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qv5lr" event={"ID":"fcc6b30b-8557-41c1-93b4-50db142d69eb","Type":"ContainerStarted","Data":"273f9cbd074b0f05ed8169267e9a4644b905be44b10f0b6888c49893d13e1ac8"}
Sep 18 18:39:28.855813 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.855793    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-118.ec2.internal" event={"ID":"396528f9074091ccc13f49651d4bd352","Type":"ContainerStarted","Data":"8abf2e28489056cee1404b4e1a178777a59e170282056fd79c3089e91ca96b49"}
Sep 18 18:39:28.857115 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.857094    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xwtjb" event={"ID":"af0b0fc3-baee-4f46-8ec0-83225a0cd5bf","Type":"ContainerStarted","Data":"150d0c9b19857c7cc0db2bdbed352d1a1533a7eb03aa9503a7e280ce982ed4f4"}
Sep 18 18:39:28.858513 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.858496    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8mcxq" event={"ID":"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52","Type":"ContainerStarted","Data":"88c4f8af749d3b1dd91c385b770e013a883a2ebbb4acaf4f0b612a00839d47c7"}
Sep 18 18:39:28.859902 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.859882    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg" event={"ID":"3ac8796c-5f58-4152-8474-88181c24214b","Type":"ContainerStarted","Data":"5cb4dbb43902864b3868298c49ddb3195bfa140d2472d3bd3b7127b208842cfe"}
Sep 18 18:39:28.861037 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.861017    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rb48x" event={"ID":"89672b2f-0680-4e2e-bdfa-66b6c41cacca","Type":"ContainerStarted","Data":"ee733601d416ee7c2ff5a26fe3bfe0398a3eb29c2787d5f5f278bd990e8335dc"}
Sep 18 18:39:28.869511 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:28.869476    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-118.ec2.internal" podStartSLOduration=1.869458608 podStartE2EDuration="1.869458608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:28.869423881 +0000 UTC m=+5.067908086" watchObservedRunningTime="2026-09-18 18:39:28.869458608 +0000 UTC m=+5.067942797"
Sep 18 18:39:29.426241 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:29.426087    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret\") pod \"global-pull-secret-syncer-2h6z8\" (UID: \"6b192a2e-3607-45c8-97aa-349c8b9503e0\") " pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:29.426241 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:29.426141    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs\") pod \"network-metrics-daemon-588ml\" (UID: \"ade7ca58-baa0-48f6-aed7-fe20bfe9461c\") " pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:29.426241 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:29.426170    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brpmq\" (UniqueName: \"kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq\") pod \"network-check-target-zbxjj\" (UID: \"f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237\") " pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:29.426563 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:29.426325    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:39:29.426563 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:29.426343    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:39:29.426563 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:29.426355    2558 projected.go:196] Error preparing data for projected volume kube-api-access-brpmq for pod openshift-network-diagnostics/network-check-target-zbxjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:29.426563 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:29.426412    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq podName:f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:31.426393074 +0000 UTC m=+7.624877272 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-brpmq" (UniqueName: "kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq") pod "network-check-target-zbxjj" (UID: "f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:29.426859 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:29.426836    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:29.426950 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:29.426889    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret podName:6b192a2e-3607-45c8-97aa-349c8b9503e0 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:31.426874072 +0000 UTC m=+7.625358254 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret") pod "global-pull-secret-syncer-2h6z8" (UID: "6b192a2e-3607-45c8-97aa-349c8b9503e0") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:29.427004 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:29.426953    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:29.427004 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:29.426986    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs podName:ade7ca58-baa0-48f6-aed7-fe20bfe9461c nodeName:}" failed. No retries permitted until 2026-09-18 18:39:31.426975533 +0000 UTC m=+7.625459729 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs") pod "network-metrics-daemon-588ml" (UID: "ade7ca58-baa0-48f6-aed7-fe20bfe9461c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:29.739863 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:29.739748    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 18:34:27 +0000 UTC" deadline="2028-04-11 07:15:17.303233079 +0000 UTC"
Sep 18 18:39:29.739863 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:29.739787    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13692h35m47.563450301s"
Sep 18 18:39:29.845868 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:29.845109    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:29.845868 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:29.845236    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:39:29.845868 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:29.845627    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:29.845868 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:29.845718    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:39:29.845868 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:29.845779    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:29.846343 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:29.846306    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:39:29.885619 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:29.884535    2558 generic.go:356] "Generic (PLEG): container finished" podID="db5792ee010cc74c32e3052501401cfb" containerID="dcfa6f9df97908ec205dba39663e0264235cdbc177c5d4f219d0123cd54eb36a" exitCode=0
Sep 18 18:39:29.885619 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:29.885430    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal" event={"ID":"db5792ee010cc74c32e3052501401cfb","Type":"ContainerDied","Data":"dcfa6f9df97908ec205dba39663e0264235cdbc177c5d4f219d0123cd54eb36a"}
Sep 18 18:39:30.899132 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:30.898381    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal" event={"ID":"db5792ee010cc74c32e3052501401cfb","Type":"ContainerStarted","Data":"76f7c945a900d4cea082d04c342ac7690a8a55bb06787ba6a7ec0b97e393489d"}
Sep 18 18:39:31.445156 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:31.445120    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret\") pod \"global-pull-secret-syncer-2h6z8\" (UID: \"6b192a2e-3607-45c8-97aa-349c8b9503e0\") " pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:31.445323 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:31.445183    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs\") pod \"network-metrics-daemon-588ml\" (UID: \"ade7ca58-baa0-48f6-aed7-fe20bfe9461c\") " pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:31.445323 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:31.445211    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brpmq\" (UniqueName: \"kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq\") pod \"network-check-target-zbxjj\" (UID: \"f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237\") " pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:31.445483 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:31.445375    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:39:31.445483 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:31.445394    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:39:31.445483 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:31.445413    2558 projected.go:196] Error preparing data for projected volume kube-api-access-brpmq for pod openshift-network-diagnostics/network-check-target-zbxjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:31.445483 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:31.445475    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq podName:f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:35.445456565 +0000 UTC m=+11.643940762 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-brpmq" (UniqueName: "kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq") pod "network-check-target-zbxjj" (UID: "f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:31.445694 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:31.445544    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:31.445694 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:31.445577    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret podName:6b192a2e-3607-45c8-97aa-349c8b9503e0 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:35.445566789 +0000 UTC m=+11.644050975 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret") pod "global-pull-secret-syncer-2h6z8" (UID: "6b192a2e-3607-45c8-97aa-349c8b9503e0") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:31.445694 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:31.445630    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:31.445694 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:31.445659    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs podName:ade7ca58-baa0-48f6-aed7-fe20bfe9461c nodeName:}" failed. No retries permitted until 2026-09-18 18:39:35.445649196 +0000 UTC m=+11.644133391 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs") pod "network-metrics-daemon-588ml" (UID: "ade7ca58-baa0-48f6-aed7-fe20bfe9461c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:31.839133 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:31.839075    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:31.839323 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:31.839218    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:31.839323 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:31.839209    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:39:31.839323 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:31.839294    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:31.839488 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:31.839426    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:39:31.839554 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:31.839491    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:39:33.838944 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:33.838900    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:33.838944 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:33.838933    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:33.839452 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:33.838904    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:33.839452 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:33.839028    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:39:33.839452 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:33.839138    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:39:33.839452 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:33.839215    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:39:35.480169 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:35.480135    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret\") pod \"global-pull-secret-syncer-2h6z8\" (UID: \"6b192a2e-3607-45c8-97aa-349c8b9503e0\") " pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:35.480639 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:35.480202    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs\") pod \"network-metrics-daemon-588ml\" (UID: \"ade7ca58-baa0-48f6-aed7-fe20bfe9461c\") " pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:35.480639 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:35.480228    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brpmq\" (UniqueName: \"kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq\") pod \"network-check-target-zbxjj\" (UID: \"f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237\") " pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:35.480639 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:35.480371    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:39:35.480639 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:35.480389    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:39:35.480639 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:35.480404    2558 projected.go:196] Error preparing data for projected volume kube-api-access-brpmq for pod openshift-network-diagnostics/network-check-target-zbxjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:35.480639 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:35.480472    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq podName:f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:43.480440825 +0000 UTC m=+19.678925023 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-brpmq" (UniqueName: "kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq") pod "network-check-target-zbxjj" (UID: "f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:35.480963 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:35.480784    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:35.480963 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:35.480786    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:35.480963 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:35.480865    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs podName:ade7ca58-baa0-48f6-aed7-fe20bfe9461c nodeName:}" failed. No retries permitted until 2026-09-18 18:39:43.480845857 +0000 UTC m=+19.679330039 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs") pod "network-metrics-daemon-588ml" (UID: "ade7ca58-baa0-48f6-aed7-fe20bfe9461c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:35.480963 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:35.480889    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret podName:6b192a2e-3607-45c8-97aa-349c8b9503e0 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:43.480873557 +0000 UTC m=+19.679357744 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret") pod "global-pull-secret-syncer-2h6z8" (UID: "6b192a2e-3607-45c8-97aa-349c8b9503e0") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:35.842316 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:35.841095    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:35.842316 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:35.841256    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:35.842316 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:35.841409    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:39:35.842316 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:35.841887    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:39:35.842316 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:35.841956    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:35.842316 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:35.842074    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:39:37.838422 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:37.838391    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:37.838849 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:37.838391    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:37.838849 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:37.838522    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:39:37.838849 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:37.838597    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:39:37.838849 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:37.838677    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:37.838849 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:37.838743    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:39:39.838631 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:39.838598    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:39.838631 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:39.838609    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:39.839155 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:39.838730    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:39:39.839155 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:39.838753    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:39.839155 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:39.838849    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:39:39.839155 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:39.838951    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:39:41.841083 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:41.841050    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:41.841083 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:41.841084    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:41.841554 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:41.841146    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:41.841554 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:41.841161    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:39:41.841554 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:41.841260    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:39:41.841554 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:41.841316    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:39:43.540288 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:43.540247    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret\") pod \"global-pull-secret-syncer-2h6z8\" (UID: \"6b192a2e-3607-45c8-97aa-349c8b9503e0\") " pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:43.540776 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:43.540298    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs\") pod \"network-metrics-daemon-588ml\" (UID: \"ade7ca58-baa0-48f6-aed7-fe20bfe9461c\") " pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:43.540776 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:43.540399    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:43.540776 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:43.540403    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:43.540776 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:43.540428    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brpmq\" (UniqueName: \"kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq\") pod \"network-check-target-zbxjj\" (UID: \"f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237\") " pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:43.540776 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:43.540459    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs podName:ade7ca58-baa0-48f6-aed7-fe20bfe9461c nodeName:}" failed. No retries permitted until 2026-09-18 18:39:59.540440908 +0000 UTC m=+35.738925113 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs") pod "network-metrics-daemon-588ml" (UID: "ade7ca58-baa0-48f6-aed7-fe20bfe9461c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:43.540776 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:43.540489    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret podName:6b192a2e-3607-45c8-97aa-349c8b9503e0 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:59.540480927 +0000 UTC m=+35.738965115 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret") pod "global-pull-secret-syncer-2h6z8" (UID: "6b192a2e-3607-45c8-97aa-349c8b9503e0") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:43.540776 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:43.540539    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:39:43.540776 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:43.540557    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:39:43.540776 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:43.540569    2558 projected.go:196] Error preparing data for projected volume kube-api-access-brpmq for pod openshift-network-diagnostics/network-check-target-zbxjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:43.540776 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:43.540622    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq podName:f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237 nodeName:}" failed. No retries permitted until 2026-09-18 18:39:59.540607759 +0000 UTC m=+35.739091955 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-brpmq" (UniqueName: "kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq") pod "network-check-target-zbxjj" (UID: "f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:43.838716 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:43.838647    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:43.838887 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:43.838647    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:43.838887 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:43.838776    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:39:43.838887 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:43.838647    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:43.839048 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:43.838924    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:39:43.839048 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:43.839025    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:39:45.839252 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:45.839231    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:45.839541 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:45.839303    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:39:45.839541 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:45.839405    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:45.839541 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:45.839523    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:39:45.839644 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:45.839553    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:45.839644 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:45.839628    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:39:45.928377 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:45.928312    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sqcfr" event={"ID":"281ef476-c8bd-4c7a-9357-5a7adb1f1ee7","Type":"ContainerStarted","Data":"1473cb61263e9df75a1e142289af978bd1f7ff6d60efa01cee55f7624d4654af"}
Sep 18 18:39:45.929601 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:45.929578    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8fj9b" event={"ID":"3bd977ef-b9c9-4380-923a-57758db726b1","Type":"ContainerStarted","Data":"b7a52d1ced1f4791b10b6c0f7471e0d13e5df7092cd230b7b5b1990339f319a1"}
Sep 18 18:39:45.930808 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:45.930788    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qv5lr" event={"ID":"fcc6b30b-8557-41c1-93b4-50db142d69eb","Type":"ContainerStarted","Data":"2ea91b562e9181ccdb3a822c97e9e113e32e421646413b6a81072ad9a7d50f71"}
Sep 18 18:39:45.943250 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:45.943205    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-118.ec2.internal" podStartSLOduration=18.943190461 podStartE2EDuration="18.943190461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:30.913566382 +0000 UTC m=+7.112050607" watchObservedRunningTime="2026-09-18 18:39:45.943190461 +0000 UTC m=+22.141674665"
Sep 18 18:39:45.943708 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:45.943682    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sqcfr" podStartSLOduration=4.030549524 podStartE2EDuration="20.943676109s" totalImagesPullingTime="16.913126585s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:45.94294351 +0000 UTC m=+22.141427733" watchObservedRunningTime="2026-09-18 18:39:45.943676109 +0000 UTC m=+22.142160317"
Sep 18 18:39:45.960605 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:45.960539    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8fj9b" podStartSLOduration=4.045055867 podStartE2EDuration="20.960524354s" totalImagesPullingTime="16.915468487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:45.957772447 +0000 UTC m=+22.156256652" watchObservedRunningTime="2026-09-18 18:39:45.960524354 +0000 UTC m=+22.159008560"
Sep 18 18:39:45.976694 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:45.976632    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qv5lr" podStartSLOduration=4.052596432 podStartE2EDuration="20.976613554s" totalImagesPullingTime="16.924017122s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:45.973560344 +0000 UTC m=+22.172044573" watchObservedRunningTime="2026-09-18 18:39:45.976613554 +0000 UTC m=+22.175097758"
Sep 18 18:39:46.057050 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:46.057010    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:46.139724 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:46.139538    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:46.935092 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:46.935053    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg" event={"ID":"3ac8796c-5f58-4152-8474-88181c24214b","Type":"ContainerStarted","Data":"799e6a7ea10d3ba89d0eb4abac2d270b01c168777e4fa9d496140967182028c8"}
Sep 18 18:39:46.936769 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:46.936740    2558 generic.go:356] "Generic (PLEG): container finished" podID="89672b2f-0680-4e2e-bdfa-66b6c41cacca" containerID="122cd8fea6e0bec811be0bfe965a68a9f794cf87ae8f8f24b91128551b87266d" exitCode=0
Sep 18 18:39:46.936902 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:46.936840    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rb48x" event={"ID":"89672b2f-0680-4e2e-bdfa-66b6c41cacca","Type":"ContainerDied","Data":"122cd8fea6e0bec811be0bfe965a68a9f794cf87ae8f8f24b91128551b87266d"}
Sep 18 18:39:46.938572 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:46.938548    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jq8qr" event={"ID":"1689135d-e785-446e-beb1-c106dd93ea55","Type":"ContainerStarted","Data":"9c150e9061e7724394e2ce1dc55d94489bb0c8bd494956d54327f2793b9e582c"}
Sep 18 18:39:46.941705 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:46.941687    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bwsg_636f0979-fe2e-49fd-86d6-31bd7961e736/ovn-acl-logging/0.log"
Sep 18 18:39:46.942121 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:46.942101    2558 generic.go:356] "Generic (PLEG): container finished" podID="636f0979-fe2e-49fd-86d6-31bd7961e736" containerID="c0be33cecc14768e94aa6326b7ab22830f36f3a2958dcc8deb0e81d5e290e0be" exitCode=1
Sep 18 18:39:46.942207 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:46.942165    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg" event={"ID":"636f0979-fe2e-49fd-86d6-31bd7961e736","Type":"ContainerStarted","Data":"ddff19bb2147a155585b86077889ce9396b776455f5e29065946b21826471fdb"}
Sep 18 18:39:46.942264 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:46.942244    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg" event={"ID":"636f0979-fe2e-49fd-86d6-31bd7961e736","Type":"ContainerStarted","Data":"d5ed3819f5f14f85221ef778bee78c8694c709d30cf97e1d6bb73d16c09d97a2"}
Sep 18 18:39:46.942314 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:46.942294    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg" event={"ID":"636f0979-fe2e-49fd-86d6-31bd7961e736","Type":"ContainerStarted","Data":"87a4a94faa71f81c89b128c42b7999319b60f5d660a6e7afa063625ce6e062c2"}
Sep 18 18:39:46.942372 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:46.942341    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg" event={"ID":"636f0979-fe2e-49fd-86d6-31bd7961e736","Type":"ContainerStarted","Data":"709c6cd74d4cf7e571a52c13553f7ba6a6e93e42851595688b5f6f19b5a71ff0"}
Sep 18 18:39:46.942420 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:46.942388    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg" event={"ID":"636f0979-fe2e-49fd-86d6-31bd7961e736","Type":"ContainerDied","Data":"c0be33cecc14768e94aa6326b7ab22830f36f3a2958dcc8deb0e81d5e290e0be"}
Sep 18 18:39:46.942465 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:46.942435    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg" event={"ID":"636f0979-fe2e-49fd-86d6-31bd7961e736","Type":"ContainerStarted","Data":"ebe1e6ec466b6d3009c403a094f976e18e721b60f08420d96089b19a0ea1baca"}
Sep 18 18:39:46.943874 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:46.943851    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8mcxq" event={"ID":"8fe8cdb9-6f30-4e99-8f6d-9610f622eb52","Type":"ContainerStarted","Data":"f7f117260db07f8223848d7eba4893864564e85c3e4ecb1a05c4307c4c1b01e7"}
Sep 18 18:39:46.970230 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:46.970196    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jq8qr" podStartSLOduration=5.036811348 podStartE2EDuration="21.970184282s" totalImagesPullingTime="16.933372934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:46.970093418 +0000 UTC m=+23.168577641" watchObservedRunningTime="2026-09-18 18:39:46.970184282 +0000 UTC m=+23.168668487"
Sep 18 18:39:46.985472 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:46.985427    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8mcxq" podStartSLOduration=4.697658424 podStartE2EDuration="21.985416785s" totalImagesPullingTime="17.287758361s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:46.985413542 +0000 UTC m=+23.183897743" watchObservedRunningTime="2026-09-18 18:39:46.985416785 +0000 UTC m=+23.183900990"
Sep 18 18:39:47.104197 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:47.104158    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:47.204672 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:47.204601    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:47.376419 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:47.376393    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8fj9b"
Sep 18 18:39:47.376881 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:47.376867    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8fj9b"
Sep 18 18:39:47.402486 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:47.402466    2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 18 18:39:47.779990 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:47.779876    2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T18:39:47.40248127Z","UUID":"e0fad9f7-afce-4c42-a47f-a75c12da56e3","Handler":null,"Name":"","Endpoint":""}
Sep 18 18:39:47.781535 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:47.781514    2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 18 18:39:47.781653 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:47.781547    2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 18 18:39:47.838853 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:47.838771    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:47.838853 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:47.838783    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:47.839080 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:47.838893    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:39:47.839080 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:47.839000    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:47.839188 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:47.838980    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:39:47.839188 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:47.839079    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:39:47.948461 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:47.948309    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xwtjb" event={"ID":"af0b0fc3-baee-4f46-8ec0-83225a0cd5bf","Type":"ContainerStarted","Data":"2271794184b345e08a83d2a72ba2fff240529d7c3a819d35139f8b4ce9f6ec66"}
Sep 18 18:39:47.951006 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:47.950972    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg" event={"ID":"3ac8796c-5f58-4152-8474-88181c24214b","Type":"ContainerStarted","Data":"cfde685efadd1d8632116e7f0ac517f0a5a18516ffa5dd86d0369a078709afbd"}
Sep 18 18:39:47.951811 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:47.951780    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8fj9b"
Sep 18 18:39:47.952262 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:47.952241    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8fj9b"
Sep 18 18:39:47.963232 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:47.963158    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xwtjb" podStartSLOduration=5.970415807 podStartE2EDuration="22.963139285s" totalImagesPullingTime="16.992723478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:47.962593648 +0000 UTC m=+24.161077879" watchObservedRunningTime="2026-09-18 18:39:47.963139285 +0000 UTC m=+24.161623493"
Sep 18 18:39:48.029073 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:48.029027    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:48.127837 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:48.127742    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:48.228534 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:48.228460    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:48.957036 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:48.957006    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bwsg_636f0979-fe2e-49fd-86d6-31bd7961e736/ovn-acl-logging/0.log"
Sep 18 18:39:48.957518 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:48.957437    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg" event={"ID":"636f0979-fe2e-49fd-86d6-31bd7961e736","Type":"ContainerStarted","Data":"0a0ed44236c7c63a82516d484b230f027ab91c93e62738716aadc55b2088bb7b"}
Sep 18 18:39:48.959718 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:48.959692    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg" event={"ID":"3ac8796c-5f58-4152-8474-88181c24214b","Type":"ContainerStarted","Data":"376880af9c519834640c2f4c6d1befa74548658dabd6951bc2b601bb51abd830"}
Sep 18 18:39:48.976091 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:48.976044    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jthg" podStartSLOduration=4.7634398860000005 podStartE2EDuration="23.976027835s" totalImagesPullingTime="19.212587949s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:48.975960438 +0000 UTC m=+25.174444662" watchObservedRunningTime="2026-09-18 18:39:48.976027835 +0000 UTC m=+25.174512041"
Sep 18 18:39:49.839047 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:49.839020    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:49.839047 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:49.839035    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:49.839270 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:49.839036    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:49.839270 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:49.839149    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:39:49.839270 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:49.839235    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:39:49.839421 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:49.839316    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:39:49.855935 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:49.855898    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:49.954280 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:49.954245    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:50.183461 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:50.183380    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:50.280689 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:50.280653    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:50.349761 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:50.349724    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:50.590556 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:50.590516    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:50.646545 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:50.646512    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:51.327745 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:51.327557    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sqcfr_281ef476-c8bd-4c7a-9357-5a7adb1f1ee7/dns-node-resolver/0.log"
Sep 18 18:39:51.838425 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:51.838346    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:51.838425 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:51.838367    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:51.838607 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:51.838348    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:51.838607 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:51.838447    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:39:51.838607 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:51.838525    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:39:51.838711 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:51.838607    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:39:51.969185 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:51.969156    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bwsg_636f0979-fe2e-49fd-86d6-31bd7961e736/ovn-acl-logging/0.log"
Sep 18 18:39:51.969577 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:51.969550    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg" event={"ID":"636f0979-fe2e-49fd-86d6-31bd7961e736","Type":"ContainerStarted","Data":"1438f1a1a291d6ca3e662a5419a0e70fe91f159d4ba531560b58409c0d7b4572"}
Sep 18 18:39:51.969890 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:51.969871    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:51.970102 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:51.969938    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:51.970102 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:51.970075    2558 scope.go:117] "RemoveContainer" containerID="c0be33cecc14768e94aa6326b7ab22830f36f3a2958dcc8deb0e81d5e290e0be"
Sep 18 18:39:51.971619 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:51.971587    2558 generic.go:356] "Generic (PLEG): container finished" podID="89672b2f-0680-4e2e-bdfa-66b6c41cacca" containerID="c2d91c3412723ec4be833f0a0b5a3e2d24220f1e317541243ac424f86f99b1d5" exitCode=0
Sep 18 18:39:51.971696 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:51.971620    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rb48x" event={"ID":"89672b2f-0680-4e2e-bdfa-66b6c41cacca","Type":"ContainerDied","Data":"c2d91c3412723ec4be833f0a0b5a3e2d24220f1e317541243ac424f86f99b1d5"}
Sep 18 18:39:51.985418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:51.985399    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:52.187725 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:52.187647    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:52.244191 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:52.244158    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:52.304379 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:52.304347    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:52.328113 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:52.328091    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qv5lr_fcc6b30b-8557-41c1-93b4-50db142d69eb/node-ca/0.log"
Sep 18 18:39:52.497074 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:52.497036    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:52.600522 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:52.600487    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:52.976848 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:52.976615    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rb48x" event={"ID":"89672b2f-0680-4e2e-bdfa-66b6c41cacca","Type":"ContainerStarted","Data":"c57d7a08427835b7745760de60953906c63e3afcebb245ea8108763c8509f5f5"}
Sep 18 18:39:52.980624 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:52.980607    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bwsg_636f0979-fe2e-49fd-86d6-31bd7961e736/ovn-acl-logging/0.log"
Sep 18 18:39:52.980987 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:52.980963    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg" event={"ID":"636f0979-fe2e-49fd-86d6-31bd7961e736","Type":"ContainerStarted","Data":"b3f49cdb6a406d9ba474fdd8ca6da0af8f1ce30eabc2c59368b7939a554fa959"}
Sep 18 18:39:52.981307 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:52.981289    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:52.996004 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:52.995984    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:39:53.032997 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:53.032933    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg" podStartSLOduration=11.078632707 podStartE2EDuration="28.032919722s" totalImagesPullingTime="16.954287015s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:39:53.029353682 +0000 UTC m=+29.227837886" watchObservedRunningTime="2026-09-18 18:39:53.032919722 +0000 UTC m=+29.231403955"
Sep 18 18:39:53.182866 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:53.182738    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:53.191988 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:53.191958    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2h6z8"]
Sep 18 18:39:53.192124 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:53.192110    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:53.192243 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:53.192220    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:39:53.195269 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:53.195247    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zbxjj"]
Sep 18 18:39:53.195408 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:53.195360    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:53.195489 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:53.195446    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:39:53.195982 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:53.195962    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-588ml"]
Sep 18 18:39:53.196087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:53.196068    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:53.196209 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:53.196155    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:39:53.651781 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:53.651741    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:53.826540 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:53.826488    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:53.984995 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:53.984924    2558 generic.go:356] "Generic (PLEG): container finished" podID="89672b2f-0680-4e2e-bdfa-66b6c41cacca" containerID="c57d7a08427835b7745760de60953906c63e3afcebb245ea8108763c8509f5f5" exitCode=0
Sep 18 18:39:53.985112 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:53.985011    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rb48x" event={"ID":"89672b2f-0680-4e2e-bdfa-66b6c41cacca","Type":"ContainerDied","Data":"c57d7a08427835b7745760de60953906c63e3afcebb245ea8108763c8509f5f5"}
Sep 18 18:39:54.110878 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:54.110845    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:54.212038 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:54.212010    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:54.357391 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:54.357356    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:54.839009 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:54.838982    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:54.839009 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:54.839009    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:54.839462 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:54.839010    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:54.839462 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:54.839121    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:39:54.839462 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:54.839193    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:39:54.839462 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:54.839274    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:39:55.354024 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:55.353988    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:55.566510 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:55.566420    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:55.854527 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:55.854432    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:55.923092 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:55.923065    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:55.985095 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:55.985069    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:55.991606 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:55.991580    2558 generic.go:356] "Generic (PLEG): container finished" podID="89672b2f-0680-4e2e-bdfa-66b6c41cacca" containerID="8ee0fad900621f627d1b4ad7df17de144bfe465890716aec839a531b5973b72b" exitCode=0
Sep 18 18:39:55.991701 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:55.991610    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rb48x" event={"ID":"89672b2f-0680-4e2e-bdfa-66b6c41cacca","Type":"ContainerDied","Data":"8ee0fad900621f627d1b4ad7df17de144bfe465890716aec839a531b5973b72b"}
Sep 18 18:39:56.046797 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:56.046774    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:56.155967 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:56.155909    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:56.754501 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:56.754470    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:56.839075 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:56.839042    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:56.839235 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:56.839050    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:56.839235 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:56.839050    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:56.839235 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:56.839206    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:39:56.839400 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:56.839332    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:39:56.839451 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:56.839422    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:39:57.147080 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:57.147041    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:57.354799 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:57.354708    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:57.405204 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:57.405147    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:57.746937 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:57.746903    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:57.948399 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:57.948364    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:58.555872 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:58.555838    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:58.620886 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:58.620849    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:58.753283 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:58.753246    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:58.838621 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:58.838546    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:58.838621 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:58.838570    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:58.838621 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:58.838584    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:58.838901 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:58.838654    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:39:58.838901 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:58.838808    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:39:58.839008 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:58.838928    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:39:58.973774 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:58.973743    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:59.154287 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:59.154199    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:59.454501 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:59.454419    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:39:59.567858 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:59.567811    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret\") pod \"global-pull-secret-syncer-2h6z8\" (UID: \"6b192a2e-3607-45c8-97aa-349c8b9503e0\") " pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:39:59.568242 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:59.567876    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs\") pod \"network-metrics-daemon-588ml\" (UID: \"ade7ca58-baa0-48f6-aed7-fe20bfe9461c\") " pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:39:59.568242 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:39:59.567914    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brpmq\" (UniqueName: \"kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq\") pod \"network-check-target-zbxjj\" (UID: \"f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237\") " pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:39:59.568242 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:59.567979    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:59.568242 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:59.568016    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:39:59.568242 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:59.568033    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:39:59.568242 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:59.568041    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret podName:6b192a2e-3607-45c8-97aa-349c8b9503e0 nodeName:}" failed. No retries permitted until 2026-09-18 18:40:31.568021796 +0000 UTC m=+67.766505978 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret") pod "global-pull-secret-syncer-2h6z8" (UID: "6b192a2e-3607-45c8-97aa-349c8b9503e0") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:39:59.568242 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:59.567980    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:59.568242 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:59.568041    2558 projected.go:196] Error preparing data for projected volume kube-api-access-brpmq for pod openshift-network-diagnostics/network-check-target-zbxjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:39:59.568242 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:59.568091    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs podName:ade7ca58-baa0-48f6-aed7-fe20bfe9461c nodeName:}" failed. No retries permitted until 2026-09-18 18:40:31.568076028 +0000 UTC m=+67.766560221 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs") pod "network-metrics-daemon-588ml" (UID: "ade7ca58-baa0-48f6-aed7-fe20bfe9461c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:39:59.568242 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:39:59.568140    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq podName:f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237 nodeName:}" failed. No retries permitted until 2026-09-18 18:40:31.568125556 +0000 UTC m=+67.766609758 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-brpmq" (UniqueName: "kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq") pod "network-check-target-zbxjj" (UID: "f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:40:00.157668 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:00.157630    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:00.554013 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:00.553802    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:00.691021 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:00.690945    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:00.790862 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:00.790831    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:00.839146 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:00.839118    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:40:00.839270 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:00.839159    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:40:00.839270 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:00.839213    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:40:00.839270 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:00.839229    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:40:00.839370 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:00.839286    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:40:00.839370 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:00.839346    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:40:00.890939 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:00.890909    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:00.991902 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:00.991870    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:01.006103 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:01.006074    2558 generic.go:356] "Generic (PLEG): container finished" podID="89672b2f-0680-4e2e-bdfa-66b6c41cacca" containerID="90b22e597ba5f37ad8d4294dd16282b0d7bcb42ddbf7a4610c1a876ba6aec6ea" exitCode=0
Sep 18 18:40:01.006226 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:01.006147    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rb48x" event={"ID":"89672b2f-0680-4e2e-bdfa-66b6c41cacca","Type":"ContainerDied","Data":"90b22e597ba5f37ad8d4294dd16282b0d7bcb42ddbf7a4610c1a876ba6aec6ea"}
Sep 18 18:40:01.044410 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:01.044383    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:01.386530 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:01.386451    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:02.011492 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:02.011460    2558 generic.go:356] "Generic (PLEG): container finished" podID="89672b2f-0680-4e2e-bdfa-66b6c41cacca" containerID="325f64e00dbbb66ea0dbf07a111e22d2841f2978fd59faf669b92a1ed728008c" exitCode=0
Sep 18 18:40:02.011857 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:02.011516    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rb48x" event={"ID":"89672b2f-0680-4e2e-bdfa-66b6c41cacca","Type":"ContainerDied","Data":"325f64e00dbbb66ea0dbf07a111e22d2841f2978fd59faf669b92a1ed728008c"}
Sep 18 18:40:02.208896 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:02.208863    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:02.352290 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:02.352205    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:02.692923 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:02.692839    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:02.838527 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:02.838497    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:40:02.838527 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:02.838519    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:40:02.838527 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:02.838510    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:40:02.838740 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:02.838624    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:40:02.838740 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:02.838663    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:40:02.838812 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:02.838732    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:40:02.955444 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:02.955364    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:03.018811 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:03.018784    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rb48x" event={"ID":"89672b2f-0680-4e2e-bdfa-66b6c41cacca","Type":"ContainerStarted","Data":"23770abea200c2364ed9895321d51238684d4160603f622b5c88830a936c8d35"}
Sep 18 18:40:03.039958 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:03.039913    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rb48x" podStartSLOduration=10.496499032 podStartE2EDuration="38.039898192s" totalImagesPullingTime="27.54339916s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:03.039357611 +0000 UTC m=+39.237841819" watchObservedRunningTime="2026-09-18 18:40:03.039898192 +0000 UTC m=+39.238382396"
Sep 18 18:40:03.550502 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:03.550470    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:03.822834 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:03.822745    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:03.887978 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:03.887947    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:03.953202 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:03.953176    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:04.170155 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:04.170077    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:04.363395 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:04.363359    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:04.839176 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:04.839141    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:40:04.839345 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:04.839142    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:40:04.839345 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:04.839141    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:40:04.839454 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:04.839235    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:40:04.839645 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:04.839597    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:40:04.839984 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:04.839958    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:40:06.838406 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:06.838373    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:40:06.838859 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:06.838375    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:40:06.838859 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:06.838469    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:40:06.838859 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:06.838373    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:40:06.838859 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:06.838552    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:40:06.838859 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:06.838660    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:40:08.698057 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:08.698014    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:08.799381 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:08.799348    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:08.838111 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:08.838089    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:40:08.838111 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:08.838101    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:40:08.838275 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:08.838089    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:40:08.838275 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:08.838179    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:40:08.838275 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:08.838245    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:40:08.838392 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:08.838316    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:40:09.150286 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:09.150250    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:09.332009 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:09.331983    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:09.392737 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:09.392702    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:09.528455 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:09.528434    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:09.591593 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:09.591571    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:10.146959 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:10.146925    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:10.337411 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:10.337374    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:10.436953 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:10.436875    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:10.752370 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:10.752329    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:10.838299 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:10.838269    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:40:10.838299 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:10.838288    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:40:10.838299 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:10.838302    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:40:10.838576 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:10.838372    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2h6z8" podUID="6b192a2e-3607-45c8-97aa-349c8b9503e0"
Sep 18 18:40:10.838576 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:10.838442    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-588ml" podUID="ade7ca58-baa0-48f6-aed7-fe20bfe9461c"
Sep 18 18:40:10.838576 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:10.838497    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbxjj" podUID="f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237"
Sep 18 18:40:11.539536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:11.539504    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:11.625706 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:11.625676    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:11.724930 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:11.724897    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:11.826029 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:11.825954    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:12.616914 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.616890    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-118.ec2.internal" event="NodeReady"
Sep 18 18:40:12.617281 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.616995    2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 18 18:40:12.664517 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.664442    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7zhgc"]
Sep 18 18:40:12.694048 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.694018    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gc7jn"]
Sep 18 18:40:12.694193 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.694139    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7zhgc"
Sep 18 18:40:12.696972 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.696955    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 18 18:40:12.700078 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.697547    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r76nj\""
Sep 18 18:40:12.700078 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.698590    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 18 18:40:12.715924 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.715903    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7zhgc"]
Sep 18 18:40:12.716015 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.715933    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gc7jn"]
Sep 18 18:40:12.716015 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.716005    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gc7jn"
Sep 18 18:40:12.718859 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.718841    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 18 18:40:12.718951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.718865    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 18 18:40:12.718951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.718865    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 18 18:40:12.719070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.718973    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-swhrv\""
Sep 18 18:40:12.719152 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.719123    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 18 18:40:12.761125 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.761100    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/25e887b0-03c3-463e-8009-9e0970a083c6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gc7jn\" (UID: \"25e887b0-03c3-463e-8009-9e0970a083c6\") " pod="openshift-insights/insights-runtime-extractor-gc7jn"
Sep 18 18:40:12.761240 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.761130    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/25e887b0-03c3-463e-8009-9e0970a083c6-data-volume\") pod \"insights-runtime-extractor-gc7jn\" (UID: \"25e887b0-03c3-463e-8009-9e0970a083c6\") " pod="openshift-insights/insights-runtime-extractor-gc7jn"
Sep 18 18:40:12.761240 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.761173    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmbw7\" (UniqueName: \"kubernetes.io/projected/25e887b0-03c3-463e-8009-9e0970a083c6-kube-api-access-zmbw7\") pod \"insights-runtime-extractor-gc7jn\" (UID: \"25e887b0-03c3-463e-8009-9e0970a083c6\") " pod="openshift-insights/insights-runtime-extractor-gc7jn"
Sep 18 18:40:12.761240 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.761228    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6973e0c-a5c1-400f-b6e1-734332e9ef35-metrics-tls\") pod \"dns-default-7zhgc\" (UID: \"b6973e0c-a5c1-400f-b6e1-734332e9ef35\") " pod="openshift-dns/dns-default-7zhgc"
Sep 18 18:40:12.761374 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.761258    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b6973e0c-a5c1-400f-b6e1-734332e9ef35-config-volume\") pod \"dns-default-7zhgc\" (UID: \"b6973e0c-a5c1-400f-b6e1-734332e9ef35\") " pod="openshift-dns/dns-default-7zhgc"
Sep 18 18:40:12.761374 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.761287    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/25e887b0-03c3-463e-8009-9e0970a083c6-crio-socket\") pod \"insights-runtime-extractor-gc7jn\" (UID: \"25e887b0-03c3-463e-8009-9e0970a083c6\") " pod="openshift-insights/insights-runtime-extractor-gc7jn"
Sep 18 18:40:12.761374 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.761310    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf6vv\" (UniqueName: \"kubernetes.io/projected/b6973e0c-a5c1-400f-b6e1-734332e9ef35-kube-api-access-cf6vv\") pod \"dns-default-7zhgc\" (UID: \"b6973e0c-a5c1-400f-b6e1-734332e9ef35\") " pod="openshift-dns/dns-default-7zhgc"
Sep 18 18:40:12.761374 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.761326    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/25e887b0-03c3-463e-8009-9e0970a083c6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gc7jn\" (UID: \"25e887b0-03c3-463e-8009-9e0970a083c6\") " pod="openshift-insights/insights-runtime-extractor-gc7jn"
Sep 18 18:40:12.761374 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.761344    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6973e0c-a5c1-400f-b6e1-734332e9ef35-tmp-dir\") pod \"dns-default-7zhgc\" (UID: \"b6973e0c-a5c1-400f-b6e1-734332e9ef35\") " pod="openshift-dns/dns-default-7zhgc"
Sep 18 18:40:12.766311 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.766292    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qfvct"]
Sep 18 18:40:12.783334 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.783314    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qfvct"]
Sep 18 18:40:12.783423 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.783412    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qfvct"
Sep 18 18:40:12.785940 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.785924    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 18 18:40:12.786060 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.786045    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 18 18:40:12.786104 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.786076    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ljbnj\""
Sep 18 18:40:12.786347 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.786331    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 18 18:40:12.791046 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.791023    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:12.838995 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.838970    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:40:12.838995 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.838993    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:40:12.839133 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.838980    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:40:12.841773 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.841744    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 18:40:12.841773 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.841751    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 18:40:12.841950 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.841776    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 18:40:12.841950 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.841809    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kbgtj\""
Sep 18 18:40:12.841950 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.841873    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 18:40:12.842160 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.842146    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5mb7g\""
Sep 18 18:40:12.852613 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.852568    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:12.862154 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.862137    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6973e0c-a5c1-400f-b6e1-734332e9ef35-metrics-tls\") pod \"dns-default-7zhgc\" (UID: \"b6973e0c-a5c1-400f-b6e1-734332e9ef35\") " pod="openshift-dns/dns-default-7zhgc"
Sep 18 18:40:12.862244 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.862165    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b6973e0c-a5c1-400f-b6e1-734332e9ef35-config-volume\") pod \"dns-default-7zhgc\" (UID: \"b6973e0c-a5c1-400f-b6e1-734332e9ef35\") " pod="openshift-dns/dns-default-7zhgc"
Sep 18 18:40:12.862244 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.862185    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/25e887b0-03c3-463e-8009-9e0970a083c6-crio-socket\") pod \"insights-runtime-extractor-gc7jn\" (UID: \"25e887b0-03c3-463e-8009-9e0970a083c6\") " pod="openshift-insights/insights-runtime-extractor-gc7jn"
Sep 18 18:40:12.862244 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.862209    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf6vv\" (UniqueName: \"kubernetes.io/projected/b6973e0c-a5c1-400f-b6e1-734332e9ef35-kube-api-access-cf6vv\") pod \"dns-default-7zhgc\" (UID: \"b6973e0c-a5c1-400f-b6e1-734332e9ef35\") " pod="openshift-dns/dns-default-7zhgc"
Sep 18 18:40:12.862244 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.862237    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-455dd\" (UniqueName: \"kubernetes.io/projected/8780960a-d4de-43a1-86f6-a227b558013f-kube-api-access-455dd\") pod \"ingress-canary-qfvct\" (UID: \"8780960a-d4de-43a1-86f6-a227b558013f\") " pod="openshift-ingress-canary/ingress-canary-qfvct"
Sep 18 18:40:12.862440 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.862263    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/25e887b0-03c3-463e-8009-9e0970a083c6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gc7jn\" (UID: \"25e887b0-03c3-463e-8009-9e0970a083c6\") " pod="openshift-insights/insights-runtime-extractor-gc7jn"
Sep 18 18:40:12.862440 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.862281    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8780960a-d4de-43a1-86f6-a227b558013f-cert\") pod \"ingress-canary-qfvct\" (UID: \"8780960a-d4de-43a1-86f6-a227b558013f\") " pod="openshift-ingress-canary/ingress-canary-qfvct"
Sep 18 18:40:12.862440 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.862303    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6973e0c-a5c1-400f-b6e1-734332e9ef35-tmp-dir\") pod \"dns-default-7zhgc\" (UID: \"b6973e0c-a5c1-400f-b6e1-734332e9ef35\") " pod="openshift-dns/dns-default-7zhgc"
Sep 18 18:40:12.862440 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.862386    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/25e887b0-03c3-463e-8009-9e0970a083c6-crio-socket\") pod \"insights-runtime-extractor-gc7jn\" (UID: \"25e887b0-03c3-463e-8009-9e0970a083c6\") " pod="openshift-insights/insights-runtime-extractor-gc7jn"
Sep 18 18:40:12.862603 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.862507    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/25e887b0-03c3-463e-8009-9e0970a083c6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gc7jn\" (UID: \"25e887b0-03c3-463e-8009-9e0970a083c6\") " pod="openshift-insights/insights-runtime-extractor-gc7jn"
Sep 18 18:40:12.862603 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.862538    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/25e887b0-03c3-463e-8009-9e0970a083c6-data-volume\") pod \"insights-runtime-extractor-gc7jn\" (UID: \"25e887b0-03c3-463e-8009-9e0970a083c6\") " pod="openshift-insights/insights-runtime-extractor-gc7jn"
Sep 18 18:40:12.862603 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.862596    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmbw7\" (UniqueName: \"kubernetes.io/projected/25e887b0-03c3-463e-8009-9e0970a083c6-kube-api-access-zmbw7\") pod \"insights-runtime-extractor-gc7jn\" (UID: \"25e887b0-03c3-463e-8009-9e0970a083c6\") " pod="openshift-insights/insights-runtime-extractor-gc7jn"
Sep 18 18:40:12.862729 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.862703    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6973e0c-a5c1-400f-b6e1-734332e9ef35-tmp-dir\") pod \"dns-default-7zhgc\" (UID: \"b6973e0c-a5c1-400f-b6e1-734332e9ef35\") " pod="openshift-dns/dns-default-7zhgc"
Sep 18 18:40:12.862887 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.862860    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b6973e0c-a5c1-400f-b6e1-734332e9ef35-config-volume\") pod \"dns-default-7zhgc\" (UID: \"b6973e0c-a5c1-400f-b6e1-734332e9ef35\") " pod="openshift-dns/dns-default-7zhgc"
Sep 18 18:40:12.862887 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.862874    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/25e887b0-03c3-463e-8009-9e0970a083c6-data-volume\") pod \"insights-runtime-extractor-gc7jn\" (UID: \"25e887b0-03c3-463e-8009-9e0970a083c6\") " pod="openshift-insights/insights-runtime-extractor-gc7jn"
Sep 18 18:40:12.863023 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.862862    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/25e887b0-03c3-463e-8009-9e0970a083c6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gc7jn\" (UID: \"25e887b0-03c3-463e-8009-9e0970a083c6\") " pod="openshift-insights/insights-runtime-extractor-gc7jn"
Sep 18 18:40:12.865972 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.865956    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6973e0c-a5c1-400f-b6e1-734332e9ef35-metrics-tls\") pod \"dns-default-7zhgc\" (UID: \"b6973e0c-a5c1-400f-b6e1-734332e9ef35\") " pod="openshift-dns/dns-default-7zhgc"
Sep 18 18:40:12.866040 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.865996    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/25e887b0-03c3-463e-8009-9e0970a083c6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gc7jn\" (UID: \"25e887b0-03c3-463e-8009-9e0970a083c6\") " pod="openshift-insights/insights-runtime-extractor-gc7jn"
Sep 18 18:40:12.870459 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.870438    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmbw7\" (UniqueName: \"kubernetes.io/projected/25e887b0-03c3-463e-8009-9e0970a083c6-kube-api-access-zmbw7\") pod \"insights-runtime-extractor-gc7jn\" (UID: \"25e887b0-03c3-463e-8009-9e0970a083c6\") " pod="openshift-insights/insights-runtime-extractor-gc7jn"
Sep 18 18:40:12.871066 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.871050    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf6vv\" (UniqueName: \"kubernetes.io/projected/b6973e0c-a5c1-400f-b6e1-734332e9ef35-kube-api-access-cf6vv\") pod \"dns-default-7zhgc\" (UID: \"b6973e0c-a5c1-400f-b6e1-734332e9ef35\") " pod="openshift-dns/dns-default-7zhgc"
Sep 18 18:40:12.963723 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.963670    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-455dd\" (UniqueName: \"kubernetes.io/projected/8780960a-d4de-43a1-86f6-a227b558013f-kube-api-access-455dd\") pod \"ingress-canary-qfvct\" (UID: \"8780960a-d4de-43a1-86f6-a227b558013f\") " pod="openshift-ingress-canary/ingress-canary-qfvct"
Sep 18 18:40:12.963723 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.963701    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8780960a-d4de-43a1-86f6-a227b558013f-cert\") pod \"ingress-canary-qfvct\" (UID: \"8780960a-d4de-43a1-86f6-a227b558013f\") " pod="openshift-ingress-canary/ingress-canary-qfvct"
Sep 18 18:40:12.965733 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.965712    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8780960a-d4de-43a1-86f6-a227b558013f-cert\") pod \"ingress-canary-qfvct\" (UID: \"8780960a-d4de-43a1-86f6-a227b558013f\") " pod="openshift-ingress-canary/ingress-canary-qfvct"
Sep 18 18:40:12.970967 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:12.970941    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-455dd\" (UniqueName: \"kubernetes.io/projected/8780960a-d4de-43a1-86f6-a227b558013f-kube-api-access-455dd\") pod \"ingress-canary-qfvct\" (UID: \"8780960a-d4de-43a1-86f6-a227b558013f\") " pod="openshift-ingress-canary/ingress-canary-qfvct"
Sep 18 18:40:13.004499 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:13.004481    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7zhgc"
Sep 18 18:40:13.023653 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:13.023633    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gc7jn"
Sep 18 18:40:13.091564 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:13.091528    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qfvct"
Sep 18 18:40:13.339059 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:13.339029    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7zhgc"]
Sep 18 18:40:13.340613 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:13.340591    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gc7jn"]
Sep 18 18:40:13.342627 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:40:13.342603    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6973e0c_a5c1_400f_b6e1_734332e9ef35.slice/crio-f10f7a4cc2f89d52e27cebe338a298fa648c1045bf0e8f88c9c9f4fd6e399b06 WatchSource:0}: Error finding container f10f7a4cc2f89d52e27cebe338a298fa648c1045bf0e8f88c9c9f4fd6e399b06: Status 404 returned error can't find the container with id f10f7a4cc2f89d52e27cebe338a298fa648c1045bf0e8f88c9c9f4fd6e399b06
Sep 18 18:40:13.344314 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:40:13.344288    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25e887b0_03c3_463e_8009_9e0970a083c6.slice/crio-6394871faace0379c8cc8680a02aa74957d2fa1bc6b204187cb70e1dde6f3f2e WatchSource:0}: Error finding container 6394871faace0379c8cc8680a02aa74957d2fa1bc6b204187cb70e1dde6f3f2e: Status 404 returned error can't find the container with id 6394871faace0379c8cc8680a02aa74957d2fa1bc6b204187cb70e1dde6f3f2e
Sep 18 18:40:13.385872 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:13.385850    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qfvct"]
Sep 18 18:40:13.388789 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:40:13.388767    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8780960a_d4de_43a1_86f6_a227b558013f.slice/crio-d51f93cc3e3200a67bb8774287fbc84844c71239a82484fc2d34701e5913b361 WatchSource:0}: Error finding container d51f93cc3e3200a67bb8774287fbc84844c71239a82484fc2d34701e5913b361: Status 404 returned error can't find the container with id d51f93cc3e3200a67bb8774287fbc84844c71239a82484fc2d34701e5913b361
Sep 18 18:40:14.045971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:14.045935    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gc7jn" event={"ID":"25e887b0-03c3-463e-8009-9e0970a083c6","Type":"ContainerStarted","Data":"957fd4d73e8c8dc05ecf9a7930c783f08bbe0490264ce2a6dd3ed151acf0626a"}
Sep 18 18:40:14.046386 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:14.045999    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gc7jn" event={"ID":"25e887b0-03c3-463e-8009-9e0970a083c6","Type":"ContainerStarted","Data":"6394871faace0379c8cc8680a02aa74957d2fa1bc6b204187cb70e1dde6f3f2e"}
Sep 18 18:40:14.047287 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:14.047263    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7zhgc" event={"ID":"b6973e0c-a5c1-400f-b6e1-734332e9ef35","Type":"ContainerStarted","Data":"f10f7a4cc2f89d52e27cebe338a298fa648c1045bf0e8f88c9c9f4fd6e399b06"}
Sep 18 18:40:14.048810 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:14.048782    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qfvct" event={"ID":"8780960a-d4de-43a1-86f6-a227b558013f","Type":"ContainerStarted","Data":"d51f93cc3e3200a67bb8774287fbc84844c71239a82484fc2d34701e5913b361"}
Sep 18 18:40:15.694556 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:15.694485    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:15.794605 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:15.794568    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:16.056943 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:16.056906    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gc7jn" event={"ID":"25e887b0-03c3-463e-8009-9e0970a083c6","Type":"ContainerStarted","Data":"341aee03ac38d1118f982312595e98b36e3aa6648379706272452ae60a6e22fc"}
Sep 18 18:40:17.063037 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.062998    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7zhgc" event={"ID":"b6973e0c-a5c1-400f-b6e1-734332e9ef35","Type":"ContainerStarted","Data":"9c4540099e9f1563b3df87cd58fd5456d669e00e952aad3d0f399de84ff5cd9c"}
Sep 18 18:40:17.063547 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.063065    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7zhgc" event={"ID":"b6973e0c-a5c1-400f-b6e1-734332e9ef35","Type":"ContainerStarted","Data":"89cf3ebb4e05068bba8196e6713bdaef56dd5219de0141b0589d05dd4f54a9a1"}
Sep 18 18:40:17.063547 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.063095    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7zhgc"
Sep 18 18:40:17.064900 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.064871    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qfvct" event={"ID":"8780960a-d4de-43a1-86f6-a227b558013f","Type":"ContainerStarted","Data":"61c1bca2d79b276eb510cd2d60c06e358b695351f4e6c90b1aa964323c3587d0"}
Sep 18 18:40:17.079041 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.079000    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7zhgc" podStartSLOduration=2.130091456 podStartE2EDuration="5.078986517s" totalImagesPullingTime="2.948895061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:17.07854225 +0000 UTC m=+53.277026454" watchObservedRunningTime="2026-09-18 18:40:17.078986517 +0000 UTC m=+53.277470721"
Sep 18 18:40:17.095507 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.095468    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qfvct" podStartSLOduration=1.9175458349999999 podStartE2EDuration="5.095455633s" totalImagesPullingTime="3.177909798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:17.094858347 +0000 UTC m=+53.293342576" watchObservedRunningTime="2026-09-18 18:40:17.095455633 +0000 UTC m=+53.293939867"
Sep 18 18:40:17.819692 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.819659    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:17.835804 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.835779    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78fd746c8b-tjcr9"]
Sep 18 18:40:17.847966 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.847938    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:17.850399 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.850379    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 18 18:40:17.850937 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.850907    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 18 18:40:17.851045 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.850941    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 18 18:40:17.851602 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.851583    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 18 18:40:17.851708 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.851686    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 18 18:40:17.851968 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.851947    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78fd746c8b-tjcr9"]
Sep 18 18:40:17.852147 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.852126    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vgbw7\""
Sep 18 18:40:17.852221 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.852166    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 18 18:40:17.852221 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.852173    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 18 18:40:17.882280 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.882259    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:17.902710 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.902690    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19a8a2ad-9601-42ef-b933-6dbf04a95758-service-ca\") pod \"console-78fd746c8b-tjcr9\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") " pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:17.902806 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.902739    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdjjz\" (UniqueName: \"kubernetes.io/projected/19a8a2ad-9601-42ef-b933-6dbf04a95758-kube-api-access-rdjjz\") pod \"console-78fd746c8b-tjcr9\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") " pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:17.902806 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.902766    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19a8a2ad-9601-42ef-b933-6dbf04a95758-console-serving-cert\") pod \"console-78fd746c8b-tjcr9\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") " pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:17.902939 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.902801    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19a8a2ad-9601-42ef-b933-6dbf04a95758-oauth-serving-cert\") pod \"console-78fd746c8b-tjcr9\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") " pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:17.902939 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.902916    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19a8a2ad-9601-42ef-b933-6dbf04a95758-console-config\") pod \"console-78fd746c8b-tjcr9\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") " pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:17.902939 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:17.902937    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19a8a2ad-9601-42ef-b933-6dbf04a95758-console-oauth-config\") pod \"console-78fd746c8b-tjcr9\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") " pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:18.001922 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.001896    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-g5kfj"]
Sep 18 18:40:18.003273 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.003248    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19a8a2ad-9601-42ef-b933-6dbf04a95758-console-serving-cert\") pod \"console-78fd746c8b-tjcr9\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") " pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:18.003377 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.003280    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19a8a2ad-9601-42ef-b933-6dbf04a95758-oauth-serving-cert\") pod \"console-78fd746c8b-tjcr9\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") " pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:18.003377 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.003323    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19a8a2ad-9601-42ef-b933-6dbf04a95758-console-config\") pod \"console-78fd746c8b-tjcr9\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") " pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:18.003483 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.003377    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19a8a2ad-9601-42ef-b933-6dbf04a95758-console-oauth-config\") pod \"console-78fd746c8b-tjcr9\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") " pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:18.003483 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.003442    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19a8a2ad-9601-42ef-b933-6dbf04a95758-service-ca\") pod \"console-78fd746c8b-tjcr9\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") " pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:18.003483 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.003474    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdjjz\" (UniqueName: \"kubernetes.io/projected/19a8a2ad-9601-42ef-b933-6dbf04a95758-kube-api-access-rdjjz\") pod \"console-78fd746c8b-tjcr9\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") " pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:18.004079 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.004055    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19a8a2ad-9601-42ef-b933-6dbf04a95758-console-config\") pod \"console-78fd746c8b-tjcr9\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") " pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:18.004170 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.004087    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19a8a2ad-9601-42ef-b933-6dbf04a95758-service-ca\") pod \"console-78fd746c8b-tjcr9\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") " pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:18.004170 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.004107    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19a8a2ad-9601-42ef-b933-6dbf04a95758-oauth-serving-cert\") pod \"console-78fd746c8b-tjcr9\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") " pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:18.005743 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.005717    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19a8a2ad-9601-42ef-b933-6dbf04a95758-console-oauth-config\") pod \"console-78fd746c8b-tjcr9\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") " pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:18.005879 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.005863    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19a8a2ad-9601-42ef-b933-6dbf04a95758-console-serving-cert\") pod \"console-78fd746c8b-tjcr9\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") " pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:18.010725 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.010704    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:18.019283 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.019259    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdjjz\" (UniqueName: \"kubernetes.io/projected/19a8a2ad-9601-42ef-b933-6dbf04a95758-kube-api-access-rdjjz\") pod \"console-78fd746c8b-tjcr9\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") " pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:18.026172 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.026153    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-g5kfj"]
Sep 18 18:40:18.026249 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.026232    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-g5kfj"
Sep 18 18:40:18.028598 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.028582    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-llvx9\""
Sep 18 18:40:18.028680 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.028661    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 18 18:40:18.070803 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.070746    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gc7jn" event={"ID":"25e887b0-03c3-463e-8009-9e0970a083c6","Type":"ContainerStarted","Data":"4b8a599c7f8c183c42ba11ee3640fd8c3f9835ac189460b98ac949ebd53e6278"}
Sep 18 18:40:18.088424 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.088387    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gc7jn" podStartSLOduration=2.295476139 podStartE2EDuration="6.088374338s" totalImagesPullingTime="3.792898199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:40:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:18.08776553 +0000 UTC m=+54.286249751" watchObservedRunningTime="2026-09-18 18:40:18.088374338 +0000 UTC m=+54.286858536"
Sep 18 18:40:18.103868 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.103845    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/47b9cc1a-d753-4187-b8db-f44aa9966171-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-g5kfj\" (UID: \"47b9cc1a-d753-4187-b8db-f44aa9966171\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-g5kfj"
Sep 18 18:40:18.119895 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.119874    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:18.157922 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.157900    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:18.205304 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.205281    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/47b9cc1a-d753-4187-b8db-f44aa9966171-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-g5kfj\" (UID: \"47b9cc1a-d753-4187-b8db-f44aa9966171\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-g5kfj"
Sep 18 18:40:18.207472 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.207450    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/47b9cc1a-d753-4187-b8db-f44aa9966171-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-g5kfj\" (UID: \"47b9cc1a-d753-4187-b8db-f44aa9966171\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-g5kfj"
Sep 18 18:40:18.227811 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.227785    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:18.335200 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.335121    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-g5kfj"
Sep 18 18:40:18.449232 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.449203    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78fd746c8b-tjcr9"]
Sep 18 18:40:18.451891 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:40:18.451864    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a8a2ad_9601_42ef_b933_6dbf04a95758.slice/crio-49c8b88e199cd3b9bd9e9baa461bb542cd2f71860c3e1e3266f05918dbe57d02 WatchSource:0}: Error finding container 49c8b88e199cd3b9bd9e9baa461bb542cd2f71860c3e1e3266f05918dbe57d02: Status 404 returned error can't find the container with id 49c8b88e199cd3b9bd9e9baa461bb542cd2f71860c3e1e3266f05918dbe57d02
Sep 18 18:40:18.629140 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.629065    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-g5kfj"]
Sep 18 18:40:18.632522 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:40:18.632497    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47b9cc1a_d753_4187_b8db_f44aa9966171.slice/crio-2dc84e9b1d0a52fab4da492d1ab499f61cff792b109d3890d001ab7fbd422a00 WatchSource:0}: Error finding container 2dc84e9b1d0a52fab4da492d1ab499f61cff792b109d3890d001ab7fbd422a00: Status 404 returned error can't find the container with id 2dc84e9b1d0a52fab4da492d1ab499f61cff792b109d3890d001ab7fbd422a00
Sep 18 18:40:18.931702 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.931622    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:18.987370 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:18.987335    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:19.075729 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:19.075611    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78fd746c8b-tjcr9" event={"ID":"19a8a2ad-9601-42ef-b933-6dbf04a95758","Type":"ContainerStarted","Data":"49c8b88e199cd3b9bd9e9baa461bb542cd2f71860c3e1e3266f05918dbe57d02"}
Sep 18 18:40:19.077095 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:19.077069    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-g5kfj" event={"ID":"47b9cc1a-d753-4187-b8db-f44aa9966171","Type":"ContainerStarted","Data":"2dc84e9b1d0a52fab4da492d1ab499f61cff792b109d3890d001ab7fbd422a00"}
Sep 18 18:40:21.960445 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:21.960412    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6867756b6b-2dqml"]
Sep 18 18:40:21.962449 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:21.962432    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:21.968526 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:21.968434    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:21.972122 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:21.972095    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 18 18:40:21.973091 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:21.973035    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6867756b6b-2dqml"]
Sep 18 18:40:22.022471 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.022426    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:22.038290 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.038268    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-console-serving-cert\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.038428 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.038312    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-console-config\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.038428 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.038331    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-service-ca\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.038428 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.038396    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-oauth-serving-cert\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.038561 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.038431    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spm62\" (UniqueName: \"kubernetes.io/projected/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-kube-api-access-spm62\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.038561 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.038477    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-console-oauth-config\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.038561 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.038498    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-trusted-ca-bundle\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.139121 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.139088    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-service-ca\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.139291 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.139126    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-oauth-serving-cert\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.139291 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.139152    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spm62\" (UniqueName: \"kubernetes.io/projected/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-kube-api-access-spm62\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.139291 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.139200    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-console-oauth-config\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.139291 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.139225    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-trusted-ca-bundle\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.139500 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.139297    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-console-serving-cert\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.139500 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.139334    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-console-config\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.140013 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.139986    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-oauth-serving-cert\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.140013 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.140000    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-service-ca\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.140181 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.140092    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-console-config\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.140181 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.140137    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-trusted-ca-bundle\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.142251 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.142224    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-console-oauth-config\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.142543 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.142521    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-console-serving-cert\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.147512 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.147475    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spm62\" (UniqueName: \"kubernetes.io/projected/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-kube-api-access-spm62\") pod \"console-6867756b6b-2dqml\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") " pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.275481 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.275435    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:22.858857 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.858766    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6867756b6b-2dqml"]
Sep 18 18:40:22.861508 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:40:22.861484    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dfd0098_ac9b_4252_b7f6_ef613c2eaca2.slice/crio-323b417946f26c05b064e88d71d0615c19115da18188af0446031f4634b88c67 WatchSource:0}: Error finding container 323b417946f26c05b064e88d71d0615c19115da18188af0446031f4634b88c67: Status 404 returned error can't find the container with id 323b417946f26c05b064e88d71d0615c19115da18188af0446031f4634b88c67
Sep 18 18:40:22.963023 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:22.962987    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:23.017705 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:23.017674    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:23.092843 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:23.092793    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78fd746c8b-tjcr9" event={"ID":"19a8a2ad-9601-42ef-b933-6dbf04a95758","Type":"ContainerStarted","Data":"465cc5940eb20c4f3283a11b2fe472707c902b857f9e610e5d8219bc92c1107c"}
Sep 18 18:40:23.094381 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:23.094352    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6867756b6b-2dqml" event={"ID":"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2","Type":"ContainerStarted","Data":"3d3bef494df7a9b69941a572a21f05b88b9223e92f28572d4a566c62ad4ed8b7"}
Sep 18 18:40:23.094510 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:23.094389    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6867756b6b-2dqml" event={"ID":"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2","Type":"ContainerStarted","Data":"323b417946f26c05b064e88d71d0615c19115da18188af0446031f4634b88c67"}
Sep 18 18:40:23.095748 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:23.095727    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-g5kfj" event={"ID":"47b9cc1a-d753-4187-b8db-f44aa9966171","Type":"ContainerStarted","Data":"3f449e8d877532b53ba603240b4a09057c5ee43ce2d6e7b549a43ec0fcb02b80"}
Sep 18 18:40:23.095902 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:23.095886    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-g5kfj"
Sep 18 18:40:23.103164 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:23.103145    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-g5kfj"
Sep 18 18:40:23.110499 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:23.110425    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78fd746c8b-tjcr9" podStartSLOduration=1.924551515 podStartE2EDuration="6.110414901s" totalImagesPullingTime="4.185863386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:23.10894982 +0000 UTC m=+59.307434040" watchObservedRunningTime="2026-09-18 18:40:23.110414901 +0000 UTC m=+59.308899104"
Sep 18 18:40:23.129552 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:23.129506    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6867756b6b-2dqml" podStartSLOduration=2.129495846 podStartE2EDuration="2.129495846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:40:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:23.129091212 +0000 UTC m=+59.327575445" watchObservedRunningTime="2026-09-18 18:40:23.129495846 +0000 UTC m=+59.327980125"
Sep 18 18:40:23.142990 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:23.142953    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-g5kfj" podStartSLOduration=2.184069618 podStartE2EDuration="6.142946698s" totalImagesPullingTime="3.95887708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:23.142315439 +0000 UTC m=+59.340799666" watchObservedRunningTime="2026-09-18 18:40:23.142946698 +0000 UTC m=+59.341430901"
Sep 18 18:40:24.003971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.003940    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:24.031061 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.031037    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv"]
Sep 18 18:40:24.033064 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.033050    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv"
Sep 18 18:40:24.036630 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.036601    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 18 18:40:24.036775 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.036661    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 18 18:40:24.036775 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.036682    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 18 18:40:24.037340 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.037324    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 18 18:40:24.037815 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.037801    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-k2mxd\""
Sep 18 18:40:24.038393 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.038374    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 18 18:40:24.046187 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.046169    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv"]
Sep 18 18:40:24.104557 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.104527    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:24.155317 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.155299    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b891bf0e-ded5-4af2-aa7a-d59098eb82de-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-bjbwv\" (UID: \"b891bf0e-ded5-4af2-aa7a-d59098eb82de\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv"
Sep 18 18:40:24.155414 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.155336    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b891bf0e-ded5-4af2-aa7a-d59098eb82de-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-bjbwv\" (UID: \"b891bf0e-ded5-4af2-aa7a-d59098eb82de\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv"
Sep 18 18:40:24.155708 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.155688    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvrtn\" (UniqueName: \"kubernetes.io/projected/b891bf0e-ded5-4af2-aa7a-d59098eb82de-kube-api-access-cvrtn\") pod \"prometheus-operator-7bd5d56d6-bjbwv\" (UID: \"b891bf0e-ded5-4af2-aa7a-d59098eb82de\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv"
Sep 18 18:40:24.155836 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.155724    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b891bf0e-ded5-4af2-aa7a-d59098eb82de-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-bjbwv\" (UID: \"b891bf0e-ded5-4af2-aa7a-d59098eb82de\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv"
Sep 18 18:40:24.257105 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.257025    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvrtn\" (UniqueName: \"kubernetes.io/projected/b891bf0e-ded5-4af2-aa7a-d59098eb82de-kube-api-access-cvrtn\") pod \"prometheus-operator-7bd5d56d6-bjbwv\" (UID: \"b891bf0e-ded5-4af2-aa7a-d59098eb82de\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv"
Sep 18 18:40:24.257105 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.257069    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b891bf0e-ded5-4af2-aa7a-d59098eb82de-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-bjbwv\" (UID: \"b891bf0e-ded5-4af2-aa7a-d59098eb82de\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv"
Sep 18 18:40:24.257256 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.257194    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b891bf0e-ded5-4af2-aa7a-d59098eb82de-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-bjbwv\" (UID: \"b891bf0e-ded5-4af2-aa7a-d59098eb82de\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv"
Sep 18 18:40:24.257256 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.257241    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b891bf0e-ded5-4af2-aa7a-d59098eb82de-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-bjbwv\" (UID: \"b891bf0e-ded5-4af2-aa7a-d59098eb82de\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv"
Sep 18 18:40:24.257360 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:24.257344    2558 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 18 18:40:24.257437 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:24.257427    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b891bf0e-ded5-4af2-aa7a-d59098eb82de-prometheus-operator-tls podName:b891bf0e-ded5-4af2-aa7a-d59098eb82de nodeName:}" failed. No retries permitted until 2026-09-18 18:40:24.757402245 +0000 UTC m=+60.955886430 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/b891bf0e-ded5-4af2-aa7a-d59098eb82de-prometheus-operator-tls") pod "prometheus-operator-7bd5d56d6-bjbwv" (UID: "b891bf0e-ded5-4af2-aa7a-d59098eb82de") : secret "prometheus-operator-tls" not found
Sep 18 18:40:24.257752 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.257736    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b891bf0e-ded5-4af2-aa7a-d59098eb82de-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-bjbwv\" (UID: \"b891bf0e-ded5-4af2-aa7a-d59098eb82de\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv"
Sep 18 18:40:24.259524 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.259494    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b891bf0e-ded5-4af2-aa7a-d59098eb82de-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-bjbwv\" (UID: \"b891bf0e-ded5-4af2-aa7a-d59098eb82de\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv"
Sep 18 18:40:24.266277 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.266258    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvrtn\" (UniqueName: \"kubernetes.io/projected/b891bf0e-ded5-4af2-aa7a-d59098eb82de-kube-api-access-cvrtn\") pod \"prometheus-operator-7bd5d56d6-bjbwv\" (UID: \"b891bf0e-ded5-4af2-aa7a-d59098eb82de\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv"
Sep 18 18:40:24.613803 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.613725    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:24.761488 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.761439    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b891bf0e-ded5-4af2-aa7a-d59098eb82de-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-bjbwv\" (UID: \"b891bf0e-ded5-4af2-aa7a-d59098eb82de\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv"
Sep 18 18:40:24.763746 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.763728    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b891bf0e-ded5-4af2-aa7a-d59098eb82de-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-bjbwv\" (UID: \"b891bf0e-ded5-4af2-aa7a-d59098eb82de\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv"
Sep 18 18:40:24.941985 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.941888    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv"
Sep 18 18:40:24.999956 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:24.999933    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7bwsg"
Sep 18 18:40:25.073794 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:25.073762    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:25.175390 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:25.175356    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:25.208772 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:25.208706    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv"]
Sep 18 18:40:25.213678 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:40:25.213655    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb891bf0e_ded5_4af2_aa7a_d59098eb82de.slice/crio-749274f6c0aafafc79e19126396916cdd869dfdb123fb07c2d4b8a0a1d620142 WatchSource:0}: Error finding container 749274f6c0aafafc79e19126396916cdd869dfdb123fb07c2d4b8a0a1d620142: Status 404 returned error can't find the container with id 749274f6c0aafafc79e19126396916cdd869dfdb123fb07c2d4b8a0a1d620142
Sep 18 18:40:25.273378 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:25.273349    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:25.373743 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:25.373714    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:25.429415 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:25.429372    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:25.967836 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:25.967767    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:26.024104 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:26.024068    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:26.108910 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:26.108878    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv" event={"ID":"b891bf0e-ded5-4af2-aa7a-d59098eb82de","Type":"ContainerStarted","Data":"749274f6c0aafafc79e19126396916cdd869dfdb123fb07c2d4b8a0a1d620142"}
Sep 18 18:40:26.731202 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:26.731113    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:27.073272 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:27.073244    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7zhgc"
Sep 18 18:40:27.114273 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:27.114241    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv" event={"ID":"b891bf0e-ded5-4af2-aa7a-d59098eb82de","Type":"ContainerStarted","Data":"e390a41369e9d3a0a4f610e8ff1302370a1d70536b030a473ef55e00d74fab46"}
Sep 18 18:40:27.114673 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:27.114291    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv" event={"ID":"b891bf0e-ded5-4af2-aa7a-d59098eb82de","Type":"ContainerStarted","Data":"063a9de2ee97dd2b16be05edc0ad704c19e114325b86ff6f02b71d19e18095b6"}
Sep 18 18:40:27.131669 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:27.131620    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-bjbwv" podStartSLOduration=1.875344994 podStartE2EDuration="3.13160213s" totalImagesPullingTime="1.256257136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:27.130636826 +0000 UTC m=+63.329121056" watchObservedRunningTime="2026-09-18 18:40:27.13160213 +0000 UTC m=+63.330086336"
Sep 18 18:40:28.158648 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:28.158620    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:28.159019 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:28.158753    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:28.163210 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:28.163191    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:29.127484 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.127457    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:29.202200 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.202167    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:29.401481 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.401366    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:29.435055 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.435026    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s"]
Sep 18 18:40:29.437049 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.437030    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s"
Sep 18 18:40:29.439618 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.439598    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 18 18:40:29.439692 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.439602    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 18 18:40:29.439692 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.439647    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-lj6kd\""
Sep 18 18:40:29.456785 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.456762    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s"]
Sep 18 18:40:29.459576 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.459557    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6c75k"]
Sep 18 18:40:29.467836 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.462060    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.467836 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.464763    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 18 18:40:29.467836 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.467583    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 18 18:40:29.467836 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.467759    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 18 18:40:29.468081 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.468005    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s6ph4\""
Sep 18 18:40:29.478836 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.478000    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-57gn7"]
Sep 18 18:40:29.480262 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.480232    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.484684 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.484650    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-bbbsb\""
Sep 18 18:40:29.485280 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.484960    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 18 18:40:29.485578 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.485039    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 18 18:40:29.485876 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.485097    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 18 18:40:29.495788 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.495769    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-57gn7"]
Sep 18 18:40:29.496992 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.496972    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e26bc53-6ab6-426b-ab64-2c369f52c9d7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-dx96s\" (UID: \"0e26bc53-6ab6-426b-ab64-2c369f52c9d7\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s"
Sep 18 18:40:29.497065 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.497004    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e26bc53-6ab6-426b-ab64-2c369f52c9d7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-dx96s\" (UID: \"0e26bc53-6ab6-426b-ab64-2c369f52c9d7\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s"
Sep 18 18:40:29.497065 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.497023    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmqwb\" (UniqueName: \"kubernetes.io/projected/0e26bc53-6ab6-426b-ab64-2c369f52c9d7-kube-api-access-zmqwb\") pod \"openshift-state-metrics-86c5bc79d6-dx96s\" (UID: \"0e26bc53-6ab6-426b-ab64-2c369f52c9d7\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s"
Sep 18 18:40:29.497138 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.497065    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e26bc53-6ab6-426b-ab64-2c369f52c9d7-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-dx96s\" (UID: \"0e26bc53-6ab6-426b-ab64-2c369f52c9d7\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s"
Sep 18 18:40:29.512260 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.512238    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:29.598338 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598314    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6d483a11-8755-4322-8843-d02a7f3d8679-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-57gn7\" (UID: \"6d483a11-8755-4322-8843-d02a7f3d8679\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.598473 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598368    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e26bc53-6ab6-426b-ab64-2c369f52c9d7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-dx96s\" (UID: \"0e26bc53-6ab6-426b-ab64-2c369f52c9d7\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s"
Sep 18 18:40:29.598473 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598399    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-node-exporter-accelerators-collector-config\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.598473 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598428    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmqwb\" (UniqueName: \"kubernetes.io/projected/0e26bc53-6ab6-426b-ab64-2c369f52c9d7-kube-api-access-zmqwb\") pod \"openshift-state-metrics-86c5bc79d6-dx96s\" (UID: \"0e26bc53-6ab6-426b-ab64-2c369f52c9d7\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s"
Sep 18 18:40:29.598473 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598464    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-metrics-client-ca\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.598626 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598497    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6d483a11-8755-4322-8843-d02a7f3d8679-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-57gn7\" (UID: \"6d483a11-8755-4322-8843-d02a7f3d8679\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.598626 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598533    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e26bc53-6ab6-426b-ab64-2c369f52c9d7-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-dx96s\" (UID: \"0e26bc53-6ab6-426b-ab64-2c369f52c9d7\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s"
Sep 18 18:40:29.598626 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598556    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-sys\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.598626 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598581    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.598626 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598614    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfnj4\" (UniqueName: \"kubernetes.io/projected/6d483a11-8755-4322-8843-d02a7f3d8679-kube-api-access-jfnj4\") pod \"kube-state-metrics-84c885658c-57gn7\" (UID: \"6d483a11-8755-4322-8843-d02a7f3d8679\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.598863 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598699    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-root\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.598863 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598740    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-node-exporter-textfile\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.598863 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598802    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-node-exporter-wtmp\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.598863 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598850    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvzxh\" (UniqueName: \"kubernetes.io/projected/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-kube-api-access-fvzxh\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.599077 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598872    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6d483a11-8755-4322-8843-d02a7f3d8679-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-57gn7\" (UID: \"6d483a11-8755-4322-8843-d02a7f3d8679\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.599077 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598904    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-node-exporter-tls\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.599077 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598921    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6d483a11-8755-4322-8843-d02a7f3d8679-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-57gn7\" (UID: \"6d483a11-8755-4322-8843-d02a7f3d8679\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.599077 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598971    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e26bc53-6ab6-426b-ab64-2c369f52c9d7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-dx96s\" (UID: \"0e26bc53-6ab6-426b-ab64-2c369f52c9d7\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s"
Sep 18 18:40:29.599077 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.598987    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d483a11-8755-4322-8843-d02a7f3d8679-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-57gn7\" (UID: \"6d483a11-8755-4322-8843-d02a7f3d8679\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.599233 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.599154    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e26bc53-6ab6-426b-ab64-2c369f52c9d7-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-dx96s\" (UID: \"0e26bc53-6ab6-426b-ab64-2c369f52c9d7\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s"
Sep 18 18:40:29.600792 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.600771    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e26bc53-6ab6-426b-ab64-2c369f52c9d7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-dx96s\" (UID: \"0e26bc53-6ab6-426b-ab64-2c369f52c9d7\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s"
Sep 18 18:40:29.601257 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.601237    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e26bc53-6ab6-426b-ab64-2c369f52c9d7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-dx96s\" (UID: \"0e26bc53-6ab6-426b-ab64-2c369f52c9d7\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s"
Sep 18 18:40:29.605488 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.605460    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:29.607651 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.607628    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmqwb\" (UniqueName: \"kubernetes.io/projected/0e26bc53-6ab6-426b-ab64-2c369f52c9d7-kube-api-access-zmqwb\") pod \"openshift-state-metrics-86c5bc79d6-dx96s\" (UID: \"0e26bc53-6ab6-426b-ab64-2c369f52c9d7\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s"
Sep 18 18:40:29.700086 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700019    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-sys\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.700086 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700051    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.700233 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700091    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfnj4\" (UniqueName: \"kubernetes.io/projected/6d483a11-8755-4322-8843-d02a7f3d8679-kube-api-access-jfnj4\") pod \"kube-state-metrics-84c885658c-57gn7\" (UID: \"6d483a11-8755-4322-8843-d02a7f3d8679\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.700233 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700130    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-root\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.700233 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700133    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-sys\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.700233 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700155    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-node-exporter-textfile\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.700233 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700216    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-root\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.700400 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700281    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-node-exporter-wtmp\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.700400 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700302    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvzxh\" (UniqueName: \"kubernetes.io/projected/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-kube-api-access-fvzxh\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.700400 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700319    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6d483a11-8755-4322-8843-d02a7f3d8679-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-57gn7\" (UID: \"6d483a11-8755-4322-8843-d02a7f3d8679\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.700400 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700365    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-node-exporter-tls\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.700400 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700394    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6d483a11-8755-4322-8843-d02a7f3d8679-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-57gn7\" (UID: \"6d483a11-8755-4322-8843-d02a7f3d8679\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.700637 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700430    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d483a11-8755-4322-8843-d02a7f3d8679-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-57gn7\" (UID: \"6d483a11-8755-4322-8843-d02a7f3d8679\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.700637 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700496    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-node-exporter-textfile\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.700637 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700501    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6d483a11-8755-4322-8843-d02a7f3d8679-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-57gn7\" (UID: \"6d483a11-8755-4322-8843-d02a7f3d8679\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.700637 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700460    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-node-exporter-wtmp\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.700637 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700546    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-node-exporter-accelerators-collector-config\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.700637 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700574    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-metrics-client-ca\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.700637 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.700615    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6d483a11-8755-4322-8843-d02a7f3d8679-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-57gn7\" (UID: \"6d483a11-8755-4322-8843-d02a7f3d8679\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.701338 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.701092    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-node-exporter-accelerators-collector-config\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.701338 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.701244    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-metrics-client-ca\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.701338 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.701247    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d483a11-8755-4322-8843-d02a7f3d8679-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-57gn7\" (UID: \"6d483a11-8755-4322-8843-d02a7f3d8679\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.701338 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.701285    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6d483a11-8755-4322-8843-d02a7f3d8679-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-57gn7\" (UID: \"6d483a11-8755-4322-8843-d02a7f3d8679\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.701647 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.701624    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6d483a11-8755-4322-8843-d02a7f3d8679-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-57gn7\" (UID: \"6d483a11-8755-4322-8843-d02a7f3d8679\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.702927 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.702907    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.703207 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.703183    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6d483a11-8755-4322-8843-d02a7f3d8679-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-57gn7\" (UID: \"6d483a11-8755-4322-8843-d02a7f3d8679\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.703320 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.703231    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-node-exporter-tls\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.703630 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.703599    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6d483a11-8755-4322-8843-d02a7f3d8679-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-57gn7\" (UID: \"6d483a11-8755-4322-8843-d02a7f3d8679\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.706405 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.706384    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:29.709532 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.709499    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvzxh\" (UniqueName: \"kubernetes.io/projected/aefada7f-11b5-4e54-b38a-2cf8ddec87eb-kube-api-access-fvzxh\") pod \"node-exporter-6c75k\" (UID: \"aefada7f-11b5-4e54-b38a-2cf8ddec87eb\") " pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.709796 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.709672    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfnj4\" (UniqueName: \"kubernetes.io/projected/6d483a11-8755-4322-8843-d02a7f3d8679-kube-api-access-jfnj4\") pod \"kube-state-metrics-84c885658c-57gn7\" (UID: \"6d483a11-8755-4322-8843-d02a7f3d8679\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.745340 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.745321    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s"
Sep 18 18:40:29.776383 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.776348    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6c75k"
Sep 18 18:40:29.786297 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:40:29.786267    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaefada7f_11b5_4e54_b38a_2cf8ddec87eb.slice/crio-5ad8f3362d946e8dd3fed8087fdc2714c28b296dda93df88bcf11de1917a403f WatchSource:0}: Error finding container 5ad8f3362d946e8dd3fed8087fdc2714c28b296dda93df88bcf11de1917a403f: Status 404 returned error can't find the container with id 5ad8f3362d946e8dd3fed8087fdc2714c28b296dda93df88bcf11de1917a403f
Sep 18 18:40:29.791760 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.791736    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7"
Sep 18 18:40:29.807155 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.807119    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:29.906964 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:29.906928    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:30.008412 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.008383    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:30.016594 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.016564    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s"]
Sep 18 18:40:30.020073 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:40:30.020052    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e26bc53_6ab6_426b_ab64_2c369f52c9d7.slice/crio-129af09f50807c78e27e3b43970ded18cb621f98aa97bcf385b99dd34d35bf34 WatchSource:0}: Error finding container 129af09f50807c78e27e3b43970ded18cb621f98aa97bcf385b99dd34d35bf34: Status 404 returned error can't find the container with id 129af09f50807c78e27e3b43970ded18cb621f98aa97bcf385b99dd34d35bf34
Sep 18 18:40:30.069103 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.069075    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-57gn7"]
Sep 18 18:40:30.072123 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:40:30.072093    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d483a11_8755_4322_8843_d02a7f3d8679.slice/crio-18ff8a3a9cc427842510204a344ba2ca1f3d2e9a116d182bdd59eb9da3b7bb0e WatchSource:0}: Error finding container 18ff8a3a9cc427842510204a344ba2ca1f3d2e9a116d182bdd59eb9da3b7bb0e: Status 404 returned error can't find the container with id 18ff8a3a9cc427842510204a344ba2ca1f3d2e9a116d182bdd59eb9da3b7bb0e
Sep 18 18:40:30.126131 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.126102    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6c75k" event={"ID":"aefada7f-11b5-4e54-b38a-2cf8ddec87eb","Type":"ContainerStarted","Data":"5ad8f3362d946e8dd3fed8087fdc2714c28b296dda93df88bcf11de1917a403f"}
Sep 18 18:40:30.127793 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.127769    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s" event={"ID":"0e26bc53-6ab6-426b-ab64-2c369f52c9d7","Type":"ContainerStarted","Data":"3465e391bf060d33aa992574286e7f4b8607a74d4ebc179b7cd201181246432c"}
Sep 18 18:40:30.127904 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.127847    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s" event={"ID":"0e26bc53-6ab6-426b-ab64-2c369f52c9d7","Type":"ContainerStarted","Data":"129af09f50807c78e27e3b43970ded18cb621f98aa97bcf385b99dd34d35bf34"}
Sep 18 18:40:30.128941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.128916    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7" event={"ID":"6d483a11-8755-4322-8843-d02a7f3d8679","Type":"ContainerStarted","Data":"18ff8a3a9cc427842510204a344ba2ca1f3d2e9a116d182bdd59eb9da3b7bb0e"}
Sep 18 18:40:30.415766 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.415656    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:30.537529 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.537498    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:30.559915 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.559871    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 18:40:30.574624 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.574599    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.578190 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.578060    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 18 18:40:30.578190 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.578069    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 18 18:40:30.578190 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.578121    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 18 18:40:30.578190 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.578138    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 18 18:40:30.578983 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.578493    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 18 18:40:30.578983 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.578754    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4bzds\""
Sep 18 18:40:30.578983 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.578844    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 18 18:40:30.578983 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.578960    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 18 18:40:30.578983 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.578980    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 18 18:40:30.579482 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.579462    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 18 18:40:30.592467 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.592445    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 18:40:30.709447 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.709358    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.709447 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.709392    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.709447 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.709422    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-config-out\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.709720 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.709464    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.709720 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.709506    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.709720 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.709597    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrqfw\" (UniqueName: \"kubernetes.io/projected/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-kube-api-access-lrqfw\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.709720 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.709652    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.709720 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.709678    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.709720 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.709715    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.709983 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.709814    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-web-config\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.709983 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.709856    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-config-volume\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.709983 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.709891    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.709983 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.709931    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.718683 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.718650    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:30.810383 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.810348    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-web-config\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.810383 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.810397    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-config-volume\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.810603 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.810435    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.810603 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.810471    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.810603 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.810524    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.810603 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.810555    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.810603 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.810587    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-config-out\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.810784 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.810619    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.810784 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.810642    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.810784 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.810673    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrqfw\" (UniqueName: \"kubernetes.io/projected/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-kube-api-access-lrqfw\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.810784 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.810713    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.810784 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.810738    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.810784 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.810773    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.811192 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.811169    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.812642 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.812612    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.813439 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.813407    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.816344 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.816297    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.816671 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.816647    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.817104 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.817061    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.817359 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.817335    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-config-volume\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.817455 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.817337    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-config-out\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.817992 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.817938    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.817992 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.817942    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-web-config\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.818126 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.818010    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.820259 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.820060    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:30.820259 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.820201    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrqfw\" (UniqueName: \"kubernetes.io/projected/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-kube-api-access-lrqfw\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.820722 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.820665    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:30.886836 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:30.886794    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:40:31.136148 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:31.136114    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s" event={"ID":"0e26bc53-6ab6-426b-ab64-2c369f52c9d7","Type":"ContainerStarted","Data":"e2711d0e321fc75c0b8ad5cb569e0ed43b18d12c330fc03e230c1368e4220d27"}
Sep 18 18:40:31.483286 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:31.483207    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:31.586530 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:31.586504    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:31.620529 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:31.620502    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs\") pod \"network-metrics-daemon-588ml\" (UID: \"ade7ca58-baa0-48f6-aed7-fe20bfe9461c\") " pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:40:31.620655 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:31.620534    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brpmq\" (UniqueName: \"kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq\") pod \"network-check-target-zbxjj\" (UID: \"f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237\") " pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:40:31.620655 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:31.620601    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret\") pod \"global-pull-secret-syncer-2h6z8\" (UID: \"6b192a2e-3607-45c8-97aa-349c8b9503e0\") " pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:40:31.623231 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:31.623213    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 18:40:31.624388 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:31.624373    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 18:40:31.624429 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:31.624378    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 18:40:31.633627 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:31.633604    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b192a2e-3607-45c8-97aa-349c8b9503e0-original-pull-secret\") pod \"global-pull-secret-syncer-2h6z8\" (UID: \"6b192a2e-3607-45c8-97aa-349c8b9503e0\") " pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:40:31.633750 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:31.633732    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ade7ca58-baa0-48f6-aed7-fe20bfe9461c-metrics-certs\") pod \"network-metrics-daemon-588ml\" (UID: \"ade7ca58-baa0-48f6-aed7-fe20bfe9461c\") " pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:40:31.634148 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:31.634132    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 18:40:31.644160 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:31.644138    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brpmq\" (UniqueName: \"kubernetes.io/projected/f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237-kube-api-access-brpmq\") pod \"network-check-target-zbxjj\" (UID: \"f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237\") " pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:40:31.747799 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:31.747777    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2h6z8"
Sep 18 18:40:31.756439 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:31.756419    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kbgtj\""
Sep 18 18:40:31.760571 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:31.760554    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5mb7g\""
Sep 18 18:40:31.764568 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:31.764552    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:40:31.768757 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:31.768747    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-588ml"
Sep 18 18:40:32.001143 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.001040    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:32.070288 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.070258    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:32.276681 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.276631    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:32.276843 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.276705    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:32.289774 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.289567    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:32.362192 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.362157    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 18:40:32.375715 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:40:32.375685    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b35f223_b9bd_4c55_95f6_b7acd9a5e63c.slice/crio-aa691bc1c0cd0853d9a7a44188e58e1ff7a95da5ec82cad9bdd4f8c8c0a377b0 WatchSource:0}: Error finding container aa691bc1c0cd0853d9a7a44188e58e1ff7a95da5ec82cad9bdd4f8c8c0a377b0: Status 404 returned error can't find the container with id aa691bc1c0cd0853d9a7a44188e58e1ff7a95da5ec82cad9bdd4f8c8c0a377b0
Sep 18 18:40:32.390653 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.390626    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zbxjj"]
Sep 18 18:40:32.391627 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.391559    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2h6z8"]
Sep 18 18:40:32.394514 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:40:32.394490    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6ff45a8_a52d_4cb3_b80a_3b9a1ee8b237.slice/crio-b32da1dd610573941c622ade7919c0ebd69d919bf7932bf8f4b26909feb1e17e WatchSource:0}: Error finding container b32da1dd610573941c622ade7919c0ebd69d919bf7932bf8f4b26909feb1e17e: Status 404 returned error can't find the container with id b32da1dd610573941c622ade7919c0ebd69d919bf7932bf8f4b26909feb1e17e
Sep 18 18:40:32.395239 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:40:32.395214    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b192a2e_3607_45c8_97aa_349c8b9503e0.slice/crio-ebb8ba03f2110e107c17341a7518be382e8f7dfa5bb94da7d80e198aa3886666 WatchSource:0}: Error finding container ebb8ba03f2110e107c17341a7518be382e8f7dfa5bb94da7d80e198aa3886666: Status 404 returned error can't find the container with id ebb8ba03f2110e107c17341a7518be382e8f7dfa5bb94da7d80e198aa3886666
Sep 18 18:40:32.396377 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.396353    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:32.403808 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.403765    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-588ml"]
Sep 18 18:40:32.406672 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:40:32.406652    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podade7ca58_baa0_48f6_aed7_fe20bfe9461c.slice/crio-c74cd8ca7a4c8d31bd9b5f4b5cacdc0e2907e2ff76d973efb8fa2cf809dc5c94 WatchSource:0}: Error finding container c74cd8ca7a4c8d31bd9b5f4b5cacdc0e2907e2ff76d973efb8fa2cf809dc5c94: Status 404 returned error can't find the container with id c74cd8ca7a4c8d31bd9b5f4b5cacdc0e2907e2ff76d973efb8fa2cf809dc5c94
Sep 18 18:40:32.493374 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.493337    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:32.512187 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.512110    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"]
Sep 18 18:40:32.515305 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.515283    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.518148 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.518128    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-51g8qssnm9tnh\""
Sep 18 18:40:32.518252 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.518139    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 18 18:40:32.518544 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.518383    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-mchtg\""
Sep 18 18:40:32.518544 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.518446    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 18 18:40:32.518544 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.518455    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 18 18:40:32.518544 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.518451    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 18 18:40:32.518544 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.518447    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 18 18:40:32.531812 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.531791    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"]
Sep 18 18:40:32.579794 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.579760    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:32.630884 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.630851    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea5ee20d-3bb1-4b88-9f09-12797e833989-metrics-client-ca\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.630884 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.630885    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ea5ee20d-3bb1-4b88-9f09-12797e833989-secret-thanos-querier-tls\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.631087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.630906    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ea5ee20d-3bb1-4b88-9f09-12797e833989-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.631087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.630954    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea5ee20d-3bb1-4b88-9f09-12797e833989-secret-grpc-tls\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.631087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.631040    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea5ee20d-3bb1-4b88-9f09-12797e833989-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.631087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.631069    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnrbt\" (UniqueName: \"kubernetes.io/projected/ea5ee20d-3bb1-4b88-9f09-12797e833989-kube-api-access-jnrbt\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.631249 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.631087    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea5ee20d-3bb1-4b88-9f09-12797e833989-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.631249 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.631119    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ea5ee20d-3bb1-4b88-9f09-12797e833989-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.732016 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.731981    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ea5ee20d-3bb1-4b88-9f09-12797e833989-secret-thanos-querier-tls\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.732016 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.732018    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ea5ee20d-3bb1-4b88-9f09-12797e833989-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.732265 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.732036    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea5ee20d-3bb1-4b88-9f09-12797e833989-secret-grpc-tls\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.732265 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.732072    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea5ee20d-3bb1-4b88-9f09-12797e833989-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.732265 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.732094    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnrbt\" (UniqueName: \"kubernetes.io/projected/ea5ee20d-3bb1-4b88-9f09-12797e833989-kube-api-access-jnrbt\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.732265 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.732115    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea5ee20d-3bb1-4b88-9f09-12797e833989-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.732265 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.732155    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ea5ee20d-3bb1-4b88-9f09-12797e833989-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.732265 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.732196    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea5ee20d-3bb1-4b88-9f09-12797e833989-metrics-client-ca\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.733021 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.732991    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea5ee20d-3bb1-4b88-9f09-12797e833989-metrics-client-ca\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.734902 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.734874    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ea5ee20d-3bb1-4b88-9f09-12797e833989-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.735020 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.734953    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea5ee20d-3bb1-4b88-9f09-12797e833989-secret-grpc-tls\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.735020 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.734996    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea5ee20d-3bb1-4b88-9f09-12797e833989-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.735182 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.735167    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea5ee20d-3bb1-4b88-9f09-12797e833989-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.735234 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.735171    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ea5ee20d-3bb1-4b88-9f09-12797e833989-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.735234 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.735216    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ea5ee20d-3bb1-4b88-9f09-12797e833989-secret-thanos-querier-tls\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.740561 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.740538    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnrbt\" (UniqueName: \"kubernetes.io/projected/ea5ee20d-3bb1-4b88-9f09-12797e833989-kube-api-access-jnrbt\") pod \"thanos-querier-549fdb4b7f-gvd8b\" (UID: \"ea5ee20d-3bb1-4b88-9f09-12797e833989\") " pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:32.825798 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:32.825717    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:33.112643 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.112502    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"]
Sep 18 18:40:33.154657 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.154601    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s" event={"ID":"0e26bc53-6ab6-426b-ab64-2c369f52c9d7","Type":"ContainerStarted","Data":"f6ac574bd26cf9813cb57fbc147e006d17871559f69b4340ef7e15e49d910b2a"}
Sep 18 18:40:33.156929 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.156862    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2h6z8" event={"ID":"6b192a2e-3607-45c8-97aa-349c8b9503e0","Type":"ContainerStarted","Data":"ebb8ba03f2110e107c17341a7518be382e8f7dfa5bb94da7d80e198aa3886666"}
Sep 18 18:40:33.159222 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.159155    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c","Type":"ContainerStarted","Data":"aa691bc1c0cd0853d9a7a44188e58e1ff7a95da5ec82cad9bdd4f8c8c0a377b0"}
Sep 18 18:40:33.163412 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.163174    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7" event={"ID":"6d483a11-8755-4322-8843-d02a7f3d8679","Type":"ContainerStarted","Data":"aae4c9b298f9eb65219a3b8f91f6c924587e272d39f53761e9c6d75a6ba123b2"}
Sep 18 18:40:33.163412 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.163230    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7" event={"ID":"6d483a11-8755-4322-8843-d02a7f3d8679","Type":"ContainerStarted","Data":"541cba33dae5a572874b378e2857b96b75f8b5204100f28206ce813fec73a378"}
Sep 18 18:40:33.163412 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.163260    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7" event={"ID":"6d483a11-8755-4322-8843-d02a7f3d8679","Type":"ContainerStarted","Data":"2774f48ae8b315ea9c812052fb19dc2aceba1b2e5951f19e6ded70499c52549d"}
Sep 18 18:40:33.165470 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.165388    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b" event={"ID":"ea5ee20d-3bb1-4b88-9f09-12797e833989","Type":"ContainerStarted","Data":"fc054e00038bbae076475216f30980311f8575223d4e34771fc3a2cc091c921d"}
Sep 18 18:40:33.167348 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.167288    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zbxjj" event={"ID":"f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237","Type":"ContainerStarted","Data":"b32da1dd610573941c622ade7919c0ebd69d919bf7932bf8f4b26909feb1e17e"}
Sep 18 18:40:33.170892 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.170278    2558 generic.go:356] "Generic (PLEG): container finished" podID="aefada7f-11b5-4e54-b38a-2cf8ddec87eb" containerID="7acb2b1338c8ade9ca6a1c476a156724b3926ab4d9fee97677d6612de052f19b" exitCode=0
Sep 18 18:40:33.170892 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.170380    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6c75k" event={"ID":"aefada7f-11b5-4e54-b38a-2cf8ddec87eb","Type":"ContainerDied","Data":"7acb2b1338c8ade9ca6a1c476a156724b3926ab4d9fee97677d6612de052f19b"}
Sep 18 18:40:33.173656 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.173106    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-dx96s" podStartSLOduration=2.205111542 podStartE2EDuration="4.173087722s" totalImagesPullingTime="1.96797618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:33.171912357 +0000 UTC m=+69.370396596" watchObservedRunningTime="2026-09-18 18:40:33.173087722 +0000 UTC m=+69.371571929"
Sep 18 18:40:33.176437 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.176203    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-588ml" event={"ID":"ade7ca58-baa0-48f6-aed7-fe20bfe9461c","Type":"ContainerStarted","Data":"c74cd8ca7a4c8d31bd9b5f4b5cacdc0e2907e2ff76d973efb8fa2cf809dc5c94"}
Sep 18 18:40:33.182338 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.182266    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:40:33.207963 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.207398    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-84c885658c-57gn7" podStartSLOduration=2.18139274 podStartE2EDuration="4.207383226s" totalImagesPullingTime="2.025990486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:33.206346009 +0000 UTC m=+69.404830229" watchObservedRunningTime="2026-09-18 18:40:33.207383226 +0000 UTC m=+69.405867431"
Sep 18 18:40:33.260598 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.259875    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78fd746c8b-tjcr9"]
Sep 18 18:40:33.546165 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.545907    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:33.854608 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.854520    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-cf66457bd-lgpm9"]
Sep 18 18:40:33.858166 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.858140    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:33.860984 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.860959    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 18 18:40:33.862007 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.861694    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 18 18:40:33.862153 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.862033    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-dwdr2\""
Sep 18 18:40:33.862153 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.862069    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 18 18:40:33.862153 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.862107    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 18 18:40:33.862320 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.862219    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-b0tl3bq8cp6cq\""
Sep 18 18:40:33.872529 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.872288    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-cf66457bd-lgpm9"]
Sep 18 18:40:33.945631 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.945596    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3ba18825-d735-4bac-9364-fc1fd3202f1f-secret-metrics-server-tls\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:33.945788 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.945642    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3ba18825-d735-4bac-9364-fc1fd3202f1f-audit-log\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:33.945788 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.945693    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6q4k\" (UniqueName: \"kubernetes.io/projected/3ba18825-d735-4bac-9364-fc1fd3202f1f-kube-api-access-s6q4k\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:33.945788 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.945725    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ba18825-d735-4bac-9364-fc1fd3202f1f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:33.945788 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.945758    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3ba18825-d735-4bac-9364-fc1fd3202f1f-metrics-server-audit-profiles\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:33.946148 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.945800    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3ba18825-d735-4bac-9364-fc1fd3202f1f-secret-metrics-server-client-certs\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:33.946148 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:33.945863    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ba18825-d735-4bac-9364-fc1fd3202f1f-client-ca-bundle\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:34.047126 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.047096    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ba18825-d735-4bac-9364-fc1fd3202f1f-client-ca-bundle\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:34.047230 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.047190    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3ba18825-d735-4bac-9364-fc1fd3202f1f-secret-metrics-server-tls\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:34.047230 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.047216    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3ba18825-d735-4bac-9364-fc1fd3202f1f-audit-log\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:34.047350 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.047244    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6q4k\" (UniqueName: \"kubernetes.io/projected/3ba18825-d735-4bac-9364-fc1fd3202f1f-kube-api-access-s6q4k\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:34.047350 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.047273    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ba18825-d735-4bac-9364-fc1fd3202f1f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:34.047350 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.047303    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3ba18825-d735-4bac-9364-fc1fd3202f1f-metrics-server-audit-profiles\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:34.047350 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.047333    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3ba18825-d735-4bac-9364-fc1fd3202f1f-secret-metrics-server-client-certs\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:34.048103 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.048079    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3ba18825-d735-4bac-9364-fc1fd3202f1f-audit-log\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:34.049603 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.049013    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ba18825-d735-4bac-9364-fc1fd3202f1f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:34.049603 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.049562    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3ba18825-d735-4bac-9364-fc1fd3202f1f-metrics-server-audit-profiles\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:34.051730 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.051702    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3ba18825-d735-4bac-9364-fc1fd3202f1f-secret-metrics-server-client-certs\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:34.052955 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.052931    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3ba18825-d735-4bac-9364-fc1fd3202f1f-secret-metrics-server-tls\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:34.053910 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.053789    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ba18825-d735-4bac-9364-fc1fd3202f1f-client-ca-bundle\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:34.057867 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.057796    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6q4k\" (UniqueName: \"kubernetes.io/projected/3ba18825-d735-4bac-9364-fc1fd3202f1f-kube-api-access-s6q4k\") pod \"metrics-server-cf66457bd-lgpm9\" (UID: \"3ba18825-d735-4bac-9364-fc1fd3202f1f\") " pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:34.174186 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.173987    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:34.185157 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.185011    2558 generic.go:356] "Generic (PLEG): container finished" podID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerID="f9765782d3ae8d6eda57c80d791e53dcd3f889756156d9dc0b8d25cdff7f957f" exitCode=0
Sep 18 18:40:34.186406 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.186311    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c","Type":"ContainerDied","Data":"f9765782d3ae8d6eda57c80d791e53dcd3f889756156d9dc0b8d25cdff7f957f"}
Sep 18 18:40:34.191541 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.190794    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6c75k" event={"ID":"aefada7f-11b5-4e54-b38a-2cf8ddec87eb","Type":"ContainerStarted","Data":"d2d4ae7fbe2912a3f90a94318fcaa8f12c9fe465d373713629cfc80d46031271"}
Sep 18 18:40:34.191541 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.190867    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6c75k" event={"ID":"aefada7f-11b5-4e54-b38a-2cf8ddec87eb","Type":"ContainerStarted","Data":"a35a18bec0f78ebdd205515f33e7fcb416c035aaa7e49ffe41bef971668e66db"}
Sep 18 18:40:34.195443 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.194255    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-588ml" event={"ID":"ade7ca58-baa0-48f6-aed7-fe20bfe9461c","Type":"ContainerStarted","Data":"cee66e23a68a2fd24cd4cfa1407f143196517904017e72b61c04555b9338e92a"}
Sep 18 18:40:34.232759 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.232673    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:34.250668 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.249256    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6c75k" podStartSLOduration=2.845118021 podStartE2EDuration="5.249237889s" totalImagesPullingTime="2.309723558s" totalInitContainerRuntime="94.39631ms" isStatefulPod=true podCreationTimestamp="2026-09-18 18:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:34.249078324 +0000 UTC m=+70.447562529" watchObservedRunningTime="2026-09-18 18:40:34.249237889 +0000 UTC m=+70.447722091"
Sep 18 18:40:34.269921 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.269809    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-hp8n9"]
Sep 18 18:40:34.274163 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.273787    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-hp8n9"
Sep 18 18:40:34.277303 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.277202    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-bzc8w\""
Sep 18 18:40:34.278333 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.278157    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 18 18:40:34.287863 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.287593    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-hp8n9"]
Sep 18 18:40:34.319353 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.319277    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:34.351464 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.351231    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jldpx\" (UniqueName: \"kubernetes.io/projected/c93117d0-c124-4cf8-837f-d9631802de61-kube-api-access-jldpx\") pod \"monitoring-plugin-5984459ff8-hp8n9\" (UID: \"c93117d0-c124-4cf8-837f-d9631802de61\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-hp8n9"
Sep 18 18:40:34.351464 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.351309    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c93117d0-c124-4cf8-837f-d9631802de61-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-hp8n9\" (UID: \"c93117d0-c124-4cf8-837f-d9631802de61\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-hp8n9"
Sep 18 18:40:34.453166 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.452975    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jldpx\" (UniqueName: \"kubernetes.io/projected/c93117d0-c124-4cf8-837f-d9631802de61-kube-api-access-jldpx\") pod \"monitoring-plugin-5984459ff8-hp8n9\" (UID: \"c93117d0-c124-4cf8-837f-d9631802de61\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-hp8n9"
Sep 18 18:40:34.453166 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.453034    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c93117d0-c124-4cf8-837f-d9631802de61-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-hp8n9\" (UID: \"c93117d0-c124-4cf8-837f-d9631802de61\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-hp8n9"
Sep 18 18:40:34.458206 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.458181    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c93117d0-c124-4cf8-837f-d9631802de61-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-hp8n9\" (UID: \"c93117d0-c124-4cf8-837f-d9631802de61\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-hp8n9"
Sep 18 18:40:34.461313 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.461263    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jldpx\" (UniqueName: \"kubernetes.io/projected/c93117d0-c124-4cf8-837f-d9631802de61-kube-api-access-jldpx\") pod \"monitoring-plugin-5984459ff8-hp8n9\" (UID: \"c93117d0-c124-4cf8-837f-d9631802de61\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-hp8n9"
Sep 18 18:40:34.464389 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.464357    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:34.487338 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.487278    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-cf66457bd-lgpm9"]
Sep 18 18:40:34.491948 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:40:34.491915    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ba18825_d735_4bac_9364_fc1fd3202f1f.slice/crio-75187598be1147d3e65e38295704245281dd4b08ac6e1087d86eba242010432a WatchSource:0}: Error finding container 75187598be1147d3e65e38295704245281dd4b08ac6e1087d86eba242010432a: Status 404 returned error can't find the container with id 75187598be1147d3e65e38295704245281dd4b08ac6e1087d86eba242010432a
Sep 18 18:40:34.564879 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.564836    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:34.588652 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.588573    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-hp8n9"
Sep 18 18:40:34.870661 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.870547    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-hp8n9"]
Sep 18 18:40:34.874640 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:40:34.874610    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc93117d0_c124_4cf8_837f_d9631802de61.slice/crio-1c50ef7ea1698cd9ff0ec138f569bd0b1c61df0c66514f1f8146679508c0d09a WatchSource:0}: Error finding container 1c50ef7ea1698cd9ff0ec138f569bd0b1c61df0c66514f1f8146679508c0d09a: Status 404 returned error can't find the container with id 1c50ef7ea1698cd9ff0ec138f569bd0b1c61df0c66514f1f8146679508c0d09a
Sep 18 18:40:34.970618 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.970583    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c9cccb589-l6j8t"]
Sep 18 18:40:34.976354 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.976326    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:34.984445 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:34.984236    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c9cccb589-l6j8t"]
Sep 18 18:40:35.049311 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.049255    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:35.059890 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.059858    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e75115a-7692-43e9-90d3-438a950b71ce-console-serving-cert\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.060036 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.059900    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-service-ca\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.060036 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.059924    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-oauth-serving-cert\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.060036 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.059965    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-trusted-ca-bundle\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.060036 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.060009    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e75115a-7692-43e9-90d3-438a950b71ce-console-oauth-config\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.060256 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.060049    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg7qd\" (UniqueName: \"kubernetes.io/projected/0e75115a-7692-43e9-90d3-438a950b71ce-kube-api-access-wg7qd\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.060256 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.060075    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-console-config\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.145538 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.145349    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:35.161231 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.160797    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e75115a-7692-43e9-90d3-438a950b71ce-console-serving-cert\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.161231 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.160862    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-service-ca\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.161231 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.160890    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-oauth-serving-cert\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.161231 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.160938    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-trusted-ca-bundle\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.161231 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.160983    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e75115a-7692-43e9-90d3-438a950b71ce-console-oauth-config\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.161231 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.161023    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg7qd\" (UniqueName: \"kubernetes.io/projected/0e75115a-7692-43e9-90d3-438a950b71ce-kube-api-access-wg7qd\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.161231 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.161051    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-console-config\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.162443 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.162128    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-console-config\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.162443 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.162154    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-trusted-ca-bundle\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.162443 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.162307    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-oauth-serving-cert\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.162443 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.162348    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-service-ca\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.166437 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.166385    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e75115a-7692-43e9-90d3-438a950b71ce-console-oauth-config\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.168325 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.168269    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e75115a-7692-43e9-90d3-438a950b71ce-console-serving-cert\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.180567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.180534    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg7qd\" (UniqueName: \"kubernetes.io/projected/0e75115a-7692-43e9-90d3-438a950b71ce-kube-api-access-wg7qd\") pod \"console-5c9cccb589-l6j8t\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") " pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.204963 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.204931    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-588ml" event={"ID":"ade7ca58-baa0-48f6-aed7-fe20bfe9461c","Type":"ContainerStarted","Data":"05dbab00baa75202abee7d4e6a37dd7d04ba0a3ccf162e303f673e965e5f4545"}
Sep 18 18:40:35.207306 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.207268    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9" event={"ID":"3ba18825-d735-4bac-9364-fc1fd3202f1f","Type":"ContainerStarted","Data":"75187598be1147d3e65e38295704245281dd4b08ac6e1087d86eba242010432a"}
Sep 18 18:40:35.209028 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.209004    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-hp8n9" event={"ID":"c93117d0-c124-4cf8-837f-d9631802de61","Type":"ContainerStarted","Data":"1c50ef7ea1698cd9ff0ec138f569bd0b1c61df0c66514f1f8146679508c0d09a"}
Sep 18 18:40:35.225700 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.224998    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-588ml" podStartSLOduration=68.61156721 podStartE2EDuration="1m10.224979432s" totalImagesPullingTime="1.613412222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:35.223982171 +0000 UTC m=+71.422466397" watchObservedRunningTime="2026-09-18 18:40:35.224979432 +0000 UTC m=+71.423463637"
Sep 18 18:40:35.291905 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.291476    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:35.324070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.323970    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:35.381503 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.381400    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:35.728917 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.728788    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 18:40:35.731734 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.731665    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:35.734020 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.733997    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.736639 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.736621    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 18 18:40:35.737300 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.736771    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 18 18:40:35.737300 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.736867    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 18 18:40:35.737588 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.736952    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 18 18:40:35.737588 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.736960    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 18 18:40:35.737953 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.737905    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 18 18:40:35.738353 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.738172    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-859rn\""
Sep 18 18:40:35.740363 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.738802    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 18 18:40:35.740693 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.740663    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-73u9f6iq2gpp9\""
Sep 18 18:40:35.740693 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.740682    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 18 18:40:35.740897 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.740793    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 18 18:40:35.740897 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.740846    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 18 18:40:35.741011 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.740957    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 18 18:40:35.741011 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.741000    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 18 18:40:35.741160 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.741035    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 18 18:40:35.742474 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.742455    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 18 18:40:35.752504 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.752419    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 18:40:35.874131 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.873914    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.874131 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.873998    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.874131 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.874056    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/577cb189-abd2-4437-a291-85010327e8a6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.874425 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.874216    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.874425 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.874254    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.874425 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.874296    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.874425 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.874327    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dppx8\" (UniqueName: \"kubernetes.io/projected/577cb189-abd2-4437-a291-85010327e8a6-kube-api-access-dppx8\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.874425 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.874358    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.874651 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.874430    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.874651 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.874457    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.874651 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.874501    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.874651 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.874524    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.874651 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.874540    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.874651 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.874560    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-config\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.874651 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.874583    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.874651 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.874608    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/577cb189-abd2-4437-a291-85010327e8a6-config-out\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.874651 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.874641    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.875096 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.874717    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.875096 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.874754    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.875096 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.874799    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-web-config\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.976064 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.975956    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/577cb189-abd2-4437-a291-85010327e8a6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.976064 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976035    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.976064 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976067    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.976332 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976152    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.976332 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976204    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dppx8\" (UniqueName: \"kubernetes.io/projected/577cb189-abd2-4437-a291-85010327e8a6-kube-api-access-dppx8\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.976332 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976238    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.976332 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976299    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.976332 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976315    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.976571 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976340    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.976571 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976358    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.976571 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976381    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.976571 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976408    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-config\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.976571 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976436    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.976571 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976463    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/577cb189-abd2-4437-a291-85010327e8a6-config-out\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.976571 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976489    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.976571 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976532    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.976571 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976551    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.977018 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976583    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-web-config\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.977018 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976612    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.977018 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.976642    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.977340 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.977314    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.978162 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.978132    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.978963 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.978696    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.979877 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.979379    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/577cb189-abd2-4437-a291-85010327e8a6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.979877 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.979674    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.979877 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.979715    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.980854 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.980460    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.981192 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.981165    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.981962 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.981777    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.985134 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.982628    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.985134 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.982643    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.985134 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.982701    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.985134 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.984942    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/577cb189-abd2-4437-a291-85010327e8a6-config-out\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.986159 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.985989    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.986159 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.985992    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.986742 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.986360    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.986742 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.986709    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.987033 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.987010    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-web-config\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.988365 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.988342    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-config\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:35.992032 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:35.992010    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dppx8\" (UniqueName: \"kubernetes.io/projected/577cb189-abd2-4437-a291-85010327e8a6-kube-api-access-dppx8\") pod \"prometheus-k8s-0\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:36.048833 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:36.048772    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:36.231013 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:36.230885    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:36.285523 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:36.285482    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:37.765670 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:37.765553    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:37.865008 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:37.864970    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:39.377128 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:39.377096    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c9cccb589-l6j8t"]
Sep 18 18:40:39.533104 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:40:39.533080    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e75115a_7692_43e9_90d3_438a950b71ce.slice/crio-c8d5de393c81498b381df9ccd9c9be84a21e5793ccabb4d2a6c005bbec6b86e2 WatchSource:0}: Error finding container c8d5de393c81498b381df9ccd9c9be84a21e5793ccabb4d2a6c005bbec6b86e2: Status 404 returned error can't find the container with id c8d5de393c81498b381df9ccd9c9be84a21e5793ccabb4d2a6c005bbec6b86e2
Sep 18 18:40:39.819164 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:39.818419    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 18:40:39.836870 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:40:39.835548    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod577cb189_abd2_4437_a291_85010327e8a6.slice/crio-b36d0c3852190f80959c07ec40053acc0346e122c7b7e45fbccd8fb2e06ca445 WatchSource:0}: Error finding container b36d0c3852190f80959c07ec40053acc0346e122c7b7e45fbccd8fb2e06ca445: Status 404 returned error can't find the container with id b36d0c3852190f80959c07ec40053acc0346e122c7b7e45fbccd8fb2e06ca445
Sep 18 18:40:40.233412 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.233333    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zbxjj" event={"ID":"f6ff45a8-a52d-4cb3-b80a-3b9a1ee8b237","Type":"ContainerStarted","Data":"750622039ebc64f6f4ac78288b7da80d60ec63bd85296e20886c0ab80ff6b0c4"}
Sep 18 18:40:40.233562 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.233543    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:40:40.234974 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.234941    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9" event={"ID":"3ba18825-d735-4bac-9364-fc1fd3202f1f","Type":"ContainerStarted","Data":"f584afa969eb5da3909734e7d2ee583c35c468d16614938a556340b2c9ce3cab"}
Sep 18 18:40:40.236379 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.236355    2558 generic.go:356] "Generic (PLEG): container finished" podID="577cb189-abd2-4437-a291-85010327e8a6" containerID="ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed" exitCode=0
Sep 18 18:40:40.236484 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.236430    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"577cb189-abd2-4437-a291-85010327e8a6","Type":"ContainerDied","Data":"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed"}
Sep 18 18:40:40.236547 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.236502    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"577cb189-abd2-4437-a291-85010327e8a6","Type":"ContainerStarted","Data":"b36d0c3852190f80959c07ec40053acc0346e122c7b7e45fbccd8fb2e06ca445"}
Sep 18 18:40:40.239813 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.239730    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c","Type":"ContainerStarted","Data":"9a09904529188db77e602c980a1eb2f9d77a00c8349f98931028b731470ac74c"}
Sep 18 18:40:40.239813 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.239777    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c","Type":"ContainerStarted","Data":"db401c07488a6ed602a18d8956218024c4eb4358553fa5eebca68ad7d340363b"}
Sep 18 18:40:40.239944 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.239832    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c","Type":"ContainerStarted","Data":"3315d42aa6ffd1c04bf7e0a7f084d655d0f29e0422b0295a858bc47cd954fad4"}
Sep 18 18:40:40.239944 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.239905    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c","Type":"ContainerStarted","Data":"554c636d2bc34792fc5cc4723cfb078c28aa465e1a78658feb3b54dc663db3c8"}
Sep 18 18:40:40.240015 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.239998    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c","Type":"ContainerStarted","Data":"e78a47072848863b62e1a944ef54339207e014e4d0e07e8b01405ef48928af79"}
Sep 18 18:40:40.241294 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.241272    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-hp8n9" event={"ID":"c93117d0-c124-4cf8-837f-d9631802de61","Type":"ContainerStarted","Data":"2edb8c9a9fb6ed50e3162e5d65af4bb35b907e4656f5ed46f4a1b4ba299f6542"}
Sep 18 18:40:40.241408 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.241320    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-hp8n9"
Sep 18 18:40:40.243113 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.243089    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c9cccb589-l6j8t" event={"ID":"0e75115a-7692-43e9-90d3-438a950b71ce","Type":"ContainerStarted","Data":"0bc44bff02ccc4a4a6cf68ed49fbae6e5ab7abc7c51f6457a0addf20dac3b82f"}
Sep 18 18:40:40.243211 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.243135    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c9cccb589-l6j8t" event={"ID":"0e75115a-7692-43e9-90d3-438a950b71ce","Type":"ContainerStarted","Data":"c8d5de393c81498b381df9ccd9c9be84a21e5793ccabb4d2a6c005bbec6b86e2"}
Sep 18 18:40:40.245219 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.245200    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b" event={"ID":"ea5ee20d-3bb1-4b88-9f09-12797e833989","Type":"ContainerStarted","Data":"5e1dc352db05a52b43b8e333b9535ebd0da7e0e17c1d46372a5377d1f06608b8"}
Sep 18 18:40:40.245320 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.245254    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b" event={"ID":"ea5ee20d-3bb1-4b88-9f09-12797e833989","Type":"ContainerStarted","Data":"95a58d15d69277ecb30fe6e167725e98892685dab17ecd1fb7c7efde58bf0e4a"}
Sep 18 18:40:40.245320 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.245286    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b" event={"ID":"ea5ee20d-3bb1-4b88-9f09-12797e833989","Type":"ContainerStarted","Data":"2eca4f8d3b692894b48acc77aaad6b04faa7e94ded1fe39a57797016a7085c82"}
Sep 18 18:40:40.246610 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.246593    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-hp8n9"
Sep 18 18:40:40.246928 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.246911    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2h6z8" event={"ID":"6b192a2e-3607-45c8-97aa-349c8b9503e0","Type":"ContainerStarted","Data":"b81a005acbd1007f66918f28bf16c23a2ee9ed6d6720f889e5a2a2d39b3e9212"}
Sep 18 18:40:40.248509 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.248476    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zbxjj" podStartSLOduration=68.146641113 podStartE2EDuration="1m15.24846526s" totalImagesPullingTime="7.101824147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:40.247913997 +0000 UTC m=+76.446398218" watchObservedRunningTime="2026-09-18 18:40:40.24846526 +0000 UTC m=+76.446949463"
Sep 18 18:40:40.277532 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.277497    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c9cccb589-l6j8t" podStartSLOduration=6.277485023 podStartE2EDuration="6.277485023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:40:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:40.276953984 +0000 UTC m=+76.475438189" watchObservedRunningTime="2026-09-18 18:40:40.277485023 +0000 UTC m=+76.475969226"
Sep 18 18:40:40.277924 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.277900    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2h6z8" podStartSLOduration=67.112698778 podStartE2EDuration="1m14.277889636s" totalImagesPullingTime="7.165190858s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:39:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:40.261463462 +0000 UTC m=+76.459947667" watchObservedRunningTime="2026-09-18 18:40:40.277889636 +0000 UTC m=+76.476373840"
Sep 18 18:40:40.318273 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.318223    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9" podStartSLOduration=2.232360292 podStartE2EDuration="7.318209263s" totalImagesPullingTime="5.085848971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:40.317116199 +0000 UTC m=+76.515600428" watchObservedRunningTime="2026-09-18 18:40:40.318209263 +0000 UTC m=+76.516693466"
Sep 18 18:40:40.331573 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.331533    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5984459ff8-hp8n9" podStartSLOduration=1.642187901 podStartE2EDuration="6.331518123s" totalImagesPullingTime="4.689330222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:40.33047905 +0000 UTC m=+76.528963256" watchObservedRunningTime="2026-09-18 18:40:40.331518123 +0000 UTC m=+76.530002327"
Sep 18 18:40:40.984934 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:40.984891    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:41.254515 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:41.254483    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c","Type":"ContainerStarted","Data":"b22faa1678aed0169d73c664ff36daafe314327cccab2dd0a417528786da4315"}
Sep 18 18:40:41.257648 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:41.257619    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b" event={"ID":"ea5ee20d-3bb1-4b88-9f09-12797e833989","Type":"ContainerStarted","Data":"66065bfed881fad96ee0ef78d5eeb94eb573c8841fa27c2bc4b09db699efad80"}
Sep 18 18:40:41.257756 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:41.257680    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b" event={"ID":"ea5ee20d-3bb1-4b88-9f09-12797e833989","Type":"ContainerStarted","Data":"de31ed3d7afd8f1823ff4aa49775b0fc4e7f6b2779e84a5a2d5ade0a464e276d"}
Sep 18 18:40:41.257756 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:41.257721    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b" event={"ID":"ea5ee20d-3bb1-4b88-9f09-12797e833989","Type":"ContainerStarted","Data":"e6464708c1c7d2a90a1675b4a6165504ee682eb61b4bf428908a3f57ef6e6578"}
Sep 18 18:40:41.282354 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:41.282314    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.658621771 podStartE2EDuration="11.282298803s" totalImagesPullingTime="7.602104916s" totalInitContainerRuntime="21.572116ms" isStatefulPod=false podCreationTimestamp="2026-09-18 18:40:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:41.279727657 +0000 UTC m=+77.478211917" watchObservedRunningTime="2026-09-18 18:40:41.282298803 +0000 UTC m=+77.480783008"
Sep 18 18:40:41.307331 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:41.305204    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b" podStartSLOduration=2.064752898 podStartE2EDuration="9.305160268s" totalImagesPullingTime="7.24040737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:40:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:41.303667062 +0000 UTC m=+77.502151285" watchObservedRunningTime="2026-09-18 18:40:41.305160268 +0000 UTC m=+77.503644476"
Sep 18 18:40:42.262177 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:42.262135    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:44.274559 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:44.274523    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"577cb189-abd2-4437-a291-85010327e8a6","Type":"ContainerStarted","Data":"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b"}
Sep 18 18:40:44.275032 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:44.274588    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"577cb189-abd2-4437-a291-85010327e8a6","Type":"ContainerStarted","Data":"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac"}
Sep 18 18:40:44.275032 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:44.274615    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"577cb189-abd2-4437-a291-85010327e8a6","Type":"ContainerStarted","Data":"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77"}
Sep 18 18:40:44.275032 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:44.274639    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"577cb189-abd2-4437-a291-85010327e8a6","Type":"ContainerStarted","Data":"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae"}
Sep 18 18:40:44.275032 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:44.274661    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"577cb189-abd2-4437-a291-85010327e8a6","Type":"ContainerStarted","Data":"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f"}
Sep 18 18:40:44.275032 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:44.274684    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"577cb189-abd2-4437-a291-85010327e8a6","Type":"ContainerStarted","Data":"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e"}
Sep 18 18:40:44.303871 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:44.303810    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.002147463 podStartE2EDuration="9.303794049s" totalImagesPullingTime="3.270310895s" totalInitContainerRuntime="31.335691ms" isStatefulPod=false podCreationTimestamp="2026-09-18 18:40:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:40:44.30195347 +0000 UTC m=+80.500437700" watchObservedRunningTime="2026-09-18 18:40:44.303794049 +0000 UTC m=+80.502278252"
Sep 18 18:40:45.291916 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:45.291877    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:45.292271 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:45.291939    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:45.296571 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:45.296549    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:45.692769 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:45.692687    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:45.746292 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:45.746265    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:46.049547 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:46.049520    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:40:46.118289 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:46.118264    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:46.176954 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:46.176924    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:46.286483 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:46.286463    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:40:46.329667 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:46.329598    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6867756b6b-2dqml"]
Sep 18 18:40:46.437869 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:46.437840    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:46.539012 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:46.538977    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:48.272773 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:48.272744    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-549fdb4b7f-gvd8b"
Sep 18 18:40:54.174593 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:54.174552    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:54.175088 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:54.174615    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:40:57.771780 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:57.771738    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:58.297396 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.297350    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78fd746c8b-tjcr9" podUID="19a8a2ad-9601-42ef-b933-6dbf04a95758" containerName="console" containerID="cri-o://465cc5940eb20c4f3283a11b2fe472707c902b857f9e610e5d8219bc92c1107c" gracePeriod=15
Sep 18 18:40:58.533530 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.533506    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:58.600269 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.600194    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19a8a2ad-9601-42ef-b933-6dbf04a95758-console-oauth-config\") pod \"19a8a2ad-9601-42ef-b933-6dbf04a95758\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") "
Sep 18 18:40:58.600269 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.600247    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19a8a2ad-9601-42ef-b933-6dbf04a95758-oauth-serving-cert\") pod \"19a8a2ad-9601-42ef-b933-6dbf04a95758\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") "
Sep 18 18:40:58.600269 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.600273    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdjjz\" (UniqueName: \"kubernetes.io/projected/19a8a2ad-9601-42ef-b933-6dbf04a95758-kube-api-access-rdjjz\") pod \"19a8a2ad-9601-42ef-b933-6dbf04a95758\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") "
Sep 18 18:40:58.600513 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.600387    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19a8a2ad-9601-42ef-b933-6dbf04a95758-service-ca\") pod \"19a8a2ad-9601-42ef-b933-6dbf04a95758\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") "
Sep 18 18:40:58.600513 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.600432    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19a8a2ad-9601-42ef-b933-6dbf04a95758-console-config\") pod \"19a8a2ad-9601-42ef-b933-6dbf04a95758\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") "
Sep 18 18:40:58.600513 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.600480    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19a8a2ad-9601-42ef-b933-6dbf04a95758-console-serving-cert\") pod \"19a8a2ad-9601-42ef-b933-6dbf04a95758\" (UID: \"19a8a2ad-9601-42ef-b933-6dbf04a95758\") "
Sep 18 18:40:58.600686 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.600660    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19a8a2ad-9601-42ef-b933-6dbf04a95758-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "19a8a2ad-9601-42ef-b933-6dbf04a95758" (UID: "19a8a2ad-9601-42ef-b933-6dbf04a95758"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:40:58.600838 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.600791    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19a8a2ad-9601-42ef-b933-6dbf04a95758-service-ca" (OuterVolumeSpecName: "service-ca") pod "19a8a2ad-9601-42ef-b933-6dbf04a95758" (UID: "19a8a2ad-9601-42ef-b933-6dbf04a95758"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:40:58.600916 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.600875    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19a8a2ad-9601-42ef-b933-6dbf04a95758-oauth-serving-cert\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:40:58.600916 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.600890    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19a8a2ad-9601-42ef-b933-6dbf04a95758-console-config" (OuterVolumeSpecName: "console-config") pod "19a8a2ad-9601-42ef-b933-6dbf04a95758" (UID: "19a8a2ad-9601-42ef-b933-6dbf04a95758"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:40:58.602571 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.602537    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19a8a2ad-9601-42ef-b933-6dbf04a95758-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "19a8a2ad-9601-42ef-b933-6dbf04a95758" (UID: "19a8a2ad-9601-42ef-b933-6dbf04a95758"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:40:58.602674 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.602572    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19a8a2ad-9601-42ef-b933-6dbf04a95758-kube-api-access-rdjjz" (OuterVolumeSpecName: "kube-api-access-rdjjz") pod "19a8a2ad-9601-42ef-b933-6dbf04a95758" (UID: "19a8a2ad-9601-42ef-b933-6dbf04a95758"). InnerVolumeSpecName "kube-api-access-rdjjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:40:58.602674 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.602594    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19a8a2ad-9601-42ef-b933-6dbf04a95758-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "19a8a2ad-9601-42ef-b933-6dbf04a95758" (UID: "19a8a2ad-9601-42ef-b933-6dbf04a95758"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:40:58.701967 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.701933    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19a8a2ad-9601-42ef-b933-6dbf04a95758-console-oauth-config\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:40:58.701967 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.701960    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdjjz\" (UniqueName: \"kubernetes.io/projected/19a8a2ad-9601-42ef-b933-6dbf04a95758-kube-api-access-rdjjz\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:40:58.701967 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.701971    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19a8a2ad-9601-42ef-b933-6dbf04a95758-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:40:58.701967 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.701979    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19a8a2ad-9601-42ef-b933-6dbf04a95758-console-config\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:40:58.702214 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:58.701987    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19a8a2ad-9601-42ef-b933-6dbf04a95758-console-serving-cert\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:40:59.338269 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:59.338232    2558 generic.go:356] "Generic (PLEG): container finished" podID="19a8a2ad-9601-42ef-b933-6dbf04a95758" containerID="465cc5940eb20c4f3283a11b2fe472707c902b857f9e610e5d8219bc92c1107c" exitCode=0
Sep 18 18:40:59.338724 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:59.338283    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78fd746c8b-tjcr9"
Sep 18 18:40:59.338724 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:59.338320    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78fd746c8b-tjcr9" event={"ID":"19a8a2ad-9601-42ef-b933-6dbf04a95758","Type":"ContainerDied","Data":"465cc5940eb20c4f3283a11b2fe472707c902b857f9e610e5d8219bc92c1107c"}
Sep 18 18:40:59.338724 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:59.338388    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78fd746c8b-tjcr9" event={"ID":"19a8a2ad-9601-42ef-b933-6dbf04a95758","Type":"ContainerDied","Data":"49c8b88e199cd3b9bd9e9baa461bb542cd2f71860c3e1e3266f05918dbe57d02"}
Sep 18 18:40:59.338724 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:59.338420    2558 scope.go:117] "RemoveContainer" containerID="465cc5940eb20c4f3283a11b2fe472707c902b857f9e610e5d8219bc92c1107c"
Sep 18 18:40:59.347139 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:59.347121    2558 scope.go:117] "RemoveContainer" containerID="465cc5940eb20c4f3283a11b2fe472707c902b857f9e610e5d8219bc92c1107c"
Sep 18 18:40:59.347389 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:40:59.347368    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"465cc5940eb20c4f3283a11b2fe472707c902b857f9e610e5d8219bc92c1107c\": container with ID starting with 465cc5940eb20c4f3283a11b2fe472707c902b857f9e610e5d8219bc92c1107c not found: ID does not exist" containerID="465cc5940eb20c4f3283a11b2fe472707c902b857f9e610e5d8219bc92c1107c"
Sep 18 18:40:59.347453 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:59.347397    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"465cc5940eb20c4f3283a11b2fe472707c902b857f9e610e5d8219bc92c1107c"} err="failed to get container status \"465cc5940eb20c4f3283a11b2fe472707c902b857f9e610e5d8219bc92c1107c\": rpc error: code = NotFound desc = could not find container \"465cc5940eb20c4f3283a11b2fe472707c902b857f9e610e5d8219bc92c1107c\": container with ID starting with 465cc5940eb20c4f3283a11b2fe472707c902b857f9e610e5d8219bc92c1107c not found: ID does not exist"
Sep 18 18:40:59.359651 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:59.359628    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78fd746c8b-tjcr9"]
Sep 18 18:40:59.363409 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:59.363389    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78fd746c8b-tjcr9"]
Sep 18 18:40:59.815040 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:59.815004    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:40:59.843460 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:59.843434    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19a8a2ad-9601-42ef-b933-6dbf04a95758" path="/var/lib/kubelet/pods/19a8a2ad-9601-42ef-b933-6dbf04a95758/volumes"
Sep 18 18:40:59.875870 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:40:59.875840    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:41:07.740054 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:07.740021    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:41:07.841336 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:07.841301    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:41:07.942095 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:07.942050    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:41:11.260910 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.260864    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zbxjj"
Sep 18 18:41:11.348762 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.348718    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6867756b6b-2dqml" podUID="3dfd0098-ac9b-4252-b7f6-ef613c2eaca2" containerName="console" containerID="cri-o://3d3bef494df7a9b69941a572a21f05b88b9223e92f28572d4a566c62ad4ed8b7" gracePeriod=15
Sep 18 18:41:11.617457 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.617433    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:41:11.712334 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.712300    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-trusted-ca-bundle\") pod \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") "
Sep 18 18:41:11.712491 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.712362    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-service-ca\") pod \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") "
Sep 18 18:41:11.712551 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.712532    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-console-serving-cert\") pod \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") "
Sep 18 18:41:11.712611 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.712599    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-console-oauth-config\") pod \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") "
Sep 18 18:41:11.712667 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.712626    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spm62\" (UniqueName: \"kubernetes.io/projected/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-kube-api-access-spm62\") pod \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") "
Sep 18 18:41:11.712667 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.712646    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-console-config\") pod \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") "
Sep 18 18:41:11.712667 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.712662    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-oauth-serving-cert\") pod \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\" (UID: \"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2\") "
Sep 18 18:41:11.712837 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.712689    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-service-ca" (OuterVolumeSpecName: "service-ca") pod "3dfd0098-ac9b-4252-b7f6-ef613c2eaca2" (UID: "3dfd0098-ac9b-4252-b7f6-ef613c2eaca2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:41:11.712837 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.712706    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3dfd0098-ac9b-4252-b7f6-ef613c2eaca2" (UID: "3dfd0098-ac9b-4252-b7f6-ef613c2eaca2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:41:11.712948 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.712897    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:11.712948 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.712915    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-trusted-ca-bundle\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:11.713106 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.713080    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3dfd0098-ac9b-4252-b7f6-ef613c2eaca2" (UID: "3dfd0098-ac9b-4252-b7f6-ef613c2eaca2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:41:11.713106 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.713084    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-console-config" (OuterVolumeSpecName: "console-config") pod "3dfd0098-ac9b-4252-b7f6-ef613c2eaca2" (UID: "3dfd0098-ac9b-4252-b7f6-ef613c2eaca2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:41:11.714959 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.714921    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-kube-api-access-spm62" (OuterVolumeSpecName: "kube-api-access-spm62") pod "3dfd0098-ac9b-4252-b7f6-ef613c2eaca2" (UID: "3dfd0098-ac9b-4252-b7f6-ef613c2eaca2"). InnerVolumeSpecName "kube-api-access-spm62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:41:11.715186 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.715165    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3dfd0098-ac9b-4252-b7f6-ef613c2eaca2" (UID: "3dfd0098-ac9b-4252-b7f6-ef613c2eaca2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:41:11.715256 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.715212    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3dfd0098-ac9b-4252-b7f6-ef613c2eaca2" (UID: "3dfd0098-ac9b-4252-b7f6-ef613c2eaca2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:41:11.813942 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.813870    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-console-serving-cert\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:11.813942 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.813898    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-console-oauth-config\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:11.813942 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.813907    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spm62\" (UniqueName: \"kubernetes.io/projected/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-kube-api-access-spm62\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:11.813942 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.813919    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-console-config\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:11.813942 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:11.813927    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2-oauth-serving-cert\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:12.394221 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:12.394188    2558 generic.go:356] "Generic (PLEG): container finished" podID="3dfd0098-ac9b-4252-b7f6-ef613c2eaca2" containerID="3d3bef494df7a9b69941a572a21f05b88b9223e92f28572d4a566c62ad4ed8b7" exitCode=0
Sep 18 18:41:12.394702 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:12.394238    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6867756b6b-2dqml"
Sep 18 18:41:12.394702 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:12.394247    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6867756b6b-2dqml" event={"ID":"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2","Type":"ContainerDied","Data":"3d3bef494df7a9b69941a572a21f05b88b9223e92f28572d4a566c62ad4ed8b7"}
Sep 18 18:41:12.394702 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:12.394296    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6867756b6b-2dqml" event={"ID":"3dfd0098-ac9b-4252-b7f6-ef613c2eaca2","Type":"ContainerDied","Data":"323b417946f26c05b064e88d71d0615c19115da18188af0446031f4634b88c67"}
Sep 18 18:41:12.394702 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:12.394322    2558 scope.go:117] "RemoveContainer" containerID="3d3bef494df7a9b69941a572a21f05b88b9223e92f28572d4a566c62ad4ed8b7"
Sep 18 18:41:12.402627 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:12.402608    2558 scope.go:117] "RemoveContainer" containerID="3d3bef494df7a9b69941a572a21f05b88b9223e92f28572d4a566c62ad4ed8b7"
Sep 18 18:41:12.402915 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:41:12.402894    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d3bef494df7a9b69941a572a21f05b88b9223e92f28572d4a566c62ad4ed8b7\": container with ID starting with 3d3bef494df7a9b69941a572a21f05b88b9223e92f28572d4a566c62ad4ed8b7 not found: ID does not exist" containerID="3d3bef494df7a9b69941a572a21f05b88b9223e92f28572d4a566c62ad4ed8b7"
Sep 18 18:41:12.403016 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:12.402922    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d3bef494df7a9b69941a572a21f05b88b9223e92f28572d4a566c62ad4ed8b7"} err="failed to get container status \"3d3bef494df7a9b69941a572a21f05b88b9223e92f28572d4a566c62ad4ed8b7\": rpc error: code = NotFound desc = could not find container \"3d3bef494df7a9b69941a572a21f05b88b9223e92f28572d4a566c62ad4ed8b7\": container with ID starting with 3d3bef494df7a9b69941a572a21f05b88b9223e92f28572d4a566c62ad4ed8b7 not found: ID does not exist"
Sep 18 18:41:12.410719 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:12.410633    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6867756b6b-2dqml"]
Sep 18 18:41:12.412369 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:12.412349    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6867756b6b-2dqml"]
Sep 18 18:41:13.843399 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:13.843350    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dfd0098-ac9b-4252-b7f6-ef613c2eaca2" path="/var/lib/kubelet/pods/3dfd0098-ac9b-4252-b7f6-ef613c2eaca2/volumes"
Sep 18 18:41:14.179920 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:14.179840    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:41:14.183749 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:14.183726    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-cf66457bd-lgpm9"
Sep 18 18:41:36.049281 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:36.049249    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:36.068608 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:36.068580    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:36.514033 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:36.514005    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:48.675809 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:48.675768    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:41:48.784575 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:48.784534    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:41:49.748671 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:49.748640    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 18:41:49.749225 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:49.748994    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="alertmanager" containerID="cri-o://e78a47072848863b62e1a944ef54339207e014e4d0e07e8b01405ef48928af79" gracePeriod=120
Sep 18 18:41:49.749225 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:49.749079    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="kube-rbac-proxy" containerID="cri-o://db401c07488a6ed602a18d8956218024c4eb4358553fa5eebca68ad7d340363b" gracePeriod=120
Sep 18 18:41:49.749225 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:49.749127    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="prom-label-proxy" containerID="cri-o://b22faa1678aed0169d73c664ff36daafe314327cccab2dd0a417528786da4315" gracePeriod=120
Sep 18 18:41:49.749225 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:49.749169    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="config-reloader" containerID="cri-o://554c636d2bc34792fc5cc4723cfb078c28aa465e1a78658feb3b54dc663db3c8" gracePeriod=120
Sep 18 18:41:49.749225 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:49.749059    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="kube-rbac-proxy-metric" containerID="cri-o://9a09904529188db77e602c980a1eb2f9d77a00c8349f98931028b731470ac74c" gracePeriod=120
Sep 18 18:41:49.749518 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:49.749062    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="kube-rbac-proxy-web" containerID="cri-o://3315d42aa6ffd1c04bf7e0a7f084d655d0f29e0422b0295a858bc47cd954fad4" gracePeriod=120
Sep 18 18:41:50.554493 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:50.554461    2558 generic.go:356] "Generic (PLEG): container finished" podID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerID="b22faa1678aed0169d73c664ff36daafe314327cccab2dd0a417528786da4315" exitCode=0
Sep 18 18:41:50.554493 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:50.554484    2558 generic.go:356] "Generic (PLEG): container finished" podID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerID="9a09904529188db77e602c980a1eb2f9d77a00c8349f98931028b731470ac74c" exitCode=0
Sep 18 18:41:50.554493 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:50.554491    2558 generic.go:356] "Generic (PLEG): container finished" podID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerID="db401c07488a6ed602a18d8956218024c4eb4358553fa5eebca68ad7d340363b" exitCode=0
Sep 18 18:41:50.554493 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:50.554496    2558 generic.go:356] "Generic (PLEG): container finished" podID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerID="554c636d2bc34792fc5cc4723cfb078c28aa465e1a78658feb3b54dc663db3c8" exitCode=0
Sep 18 18:41:50.554493 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:50.554503    2558 generic.go:356] "Generic (PLEG): container finished" podID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerID="e78a47072848863b62e1a944ef54339207e014e4d0e07e8b01405ef48928af79" exitCode=0
Sep 18 18:41:50.554770 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:50.554528    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c","Type":"ContainerDied","Data":"b22faa1678aed0169d73c664ff36daafe314327cccab2dd0a417528786da4315"}
Sep 18 18:41:50.554770 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:50.554590    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c","Type":"ContainerDied","Data":"9a09904529188db77e602c980a1eb2f9d77a00c8349f98931028b731470ac74c"}
Sep 18 18:41:50.554770 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:50.554616    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c","Type":"ContainerDied","Data":"db401c07488a6ed602a18d8956218024c4eb4358553fa5eebca68ad7d340363b"}
Sep 18 18:41:50.554770 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:50.554640    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c","Type":"ContainerDied","Data":"554c636d2bc34792fc5cc4723cfb078c28aa465e1a78658feb3b54dc663db3c8"}
Sep 18 18:41:50.554770 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:50.554663    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c","Type":"ContainerDied","Data":"e78a47072848863b62e1a944ef54339207e014e4d0e07e8b01405ef48928af79"}
Sep 18 18:41:50.987863 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:50.987842    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.084706 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.084637    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-alertmanager-main-db\") pod \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") "
Sep 18 18:41:51.084706 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.084678    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-config-volume\") pod \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") "
Sep 18 18:41:51.084933 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.084715    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-kube-rbac-proxy-web\") pod \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") "
Sep 18 18:41:51.084933 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.084749    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-web-config\") pod \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") "
Sep 18 18:41:51.084933 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.084776    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-main-tls\") pod \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") "
Sep 18 18:41:51.084933 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.084844    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-config-out\") pod \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") "
Sep 18 18:41:51.084933 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.084870    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-alertmanager-trusted-ca-bundle\") pod \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") "
Sep 18 18:41:51.084933 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.084908    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-cluster-tls-config\") pod \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") "
Sep 18 18:41:51.085277 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.084953    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-tls-assets\") pod \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") "
Sep 18 18:41:51.085277 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.084992    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" (UID: "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:41:51.085277 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.085013    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-metrics-client-ca\") pod \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") "
Sep 18 18:41:51.085277 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.085050    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") "
Sep 18 18:41:51.085277 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.085080    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrqfw\" (UniqueName: \"kubernetes.io/projected/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-kube-api-access-lrqfw\") pod \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") "
Sep 18 18:41:51.085277 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.085120    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-kube-rbac-proxy\") pod \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\" (UID: \"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c\") "
Sep 18 18:41:51.085637 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.085396    2558 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-alertmanager-main-db\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:51.085637 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.085467    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" (UID: "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:41:51.086093 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.086062    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" (UID: "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:41:51.087910 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.087874    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" (UID: "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:41:51.088339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.088309    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" (UID: "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:41:51.088893 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.088852    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" (UID: "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:41:51.088995 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.088888    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" (UID: "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:41:51.089069 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.088997    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" (UID: "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:41:51.089133 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.089086    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-config-volume" (OuterVolumeSpecName: "config-volume") pod "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" (UID: "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:41:51.089196 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.089164    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-config-out" (OuterVolumeSpecName: "config-out") pod "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" (UID: "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:41:51.089746 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.089722    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-kube-api-access-lrqfw" (OuterVolumeSpecName: "kube-api-access-lrqfw") pod "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" (UID: "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c"). InnerVolumeSpecName "kube-api-access-lrqfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:41:51.092627 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.092498    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" (UID: "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:41:51.098966 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.098946    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-web-config" (OuterVolumeSpecName: "web-config") pod "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" (UID: "9b35f223-b9bd-4c55-95f6-b7acd9a5e63c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:41:51.185792 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.185766    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-metrics-client-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:51.185792 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.185791    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:51.186001 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.185807    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrqfw\" (UniqueName: \"kubernetes.io/projected/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-kube-api-access-lrqfw\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:51.186001 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.185840    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:51.186001 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.185853    2558 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:51.186001 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.185866    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:51.186001 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.185877    2558 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-web-config\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:51.186001 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.185888    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-secret-alertmanager-main-tls\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:51.186001 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.185899    2558 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-config-out\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:51.186001 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.185913    2558 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:51.186001 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.185926    2558 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-cluster-tls-config\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:51.186001 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.185939    2558 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c-tls-assets\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:51.562411 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.562378    2558 generic.go:356] "Generic (PLEG): container finished" podID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerID="3315d42aa6ffd1c04bf7e0a7f084d655d0f29e0422b0295a858bc47cd954fad4" exitCode=0
Sep 18 18:41:51.562556 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.562459    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.562556 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.562465    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c","Type":"ContainerDied","Data":"3315d42aa6ffd1c04bf7e0a7f084d655d0f29e0422b0295a858bc47cd954fad4"}
Sep 18 18:41:51.562556 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.562533    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b35f223-b9bd-4c55-95f6-b7acd9a5e63c","Type":"ContainerDied","Data":"aa691bc1c0cd0853d9a7a44188e58e1ff7a95da5ec82cad9bdd4f8c8c0a377b0"}
Sep 18 18:41:51.562659 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.562569    2558 scope.go:117] "RemoveContainer" containerID="b22faa1678aed0169d73c664ff36daafe314327cccab2dd0a417528786da4315"
Sep 18 18:41:51.570043 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.570025    2558 scope.go:117] "RemoveContainer" containerID="9a09904529188db77e602c980a1eb2f9d77a00c8349f98931028b731470ac74c"
Sep 18 18:41:51.576748 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.576731    2558 scope.go:117] "RemoveContainer" containerID="db401c07488a6ed602a18d8956218024c4eb4358553fa5eebca68ad7d340363b"
Sep 18 18:41:51.583057 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.583018    2558 scope.go:117] "RemoveContainer" containerID="3315d42aa6ffd1c04bf7e0a7f084d655d0f29e0422b0295a858bc47cd954fad4"
Sep 18 18:41:51.586572 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.586552    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 18:41:51.590908 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.590885    2558 scope.go:117] "RemoveContainer" containerID="554c636d2bc34792fc5cc4723cfb078c28aa465e1a78658feb3b54dc663db3c8"
Sep 18 18:41:51.591075 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.591055    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 18:41:51.597499 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.597482    2558 scope.go:117] "RemoveContainer" containerID="e78a47072848863b62e1a944ef54339207e014e4d0e07e8b01405ef48928af79"
Sep 18 18:41:51.604114 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.604099    2558 scope.go:117] "RemoveContainer" containerID="f9765782d3ae8d6eda57c80d791e53dcd3f889756156d9dc0b8d25cdff7f957f"
Sep 18 18:41:51.610813 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.610795    2558 scope.go:117] "RemoveContainer" containerID="b22faa1678aed0169d73c664ff36daafe314327cccab2dd0a417528786da4315"
Sep 18 18:41:51.611081 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:41:51.611062    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b22faa1678aed0169d73c664ff36daafe314327cccab2dd0a417528786da4315\": container with ID starting with b22faa1678aed0169d73c664ff36daafe314327cccab2dd0a417528786da4315 not found: ID does not exist" containerID="b22faa1678aed0169d73c664ff36daafe314327cccab2dd0a417528786da4315"
Sep 18 18:41:51.611146 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.611089    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b22faa1678aed0169d73c664ff36daafe314327cccab2dd0a417528786da4315"} err="failed to get container status \"b22faa1678aed0169d73c664ff36daafe314327cccab2dd0a417528786da4315\": rpc error: code = NotFound desc = could not find container \"b22faa1678aed0169d73c664ff36daafe314327cccab2dd0a417528786da4315\": container with ID starting with b22faa1678aed0169d73c664ff36daafe314327cccab2dd0a417528786da4315 not found: ID does not exist"
Sep 18 18:41:51.611146 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.611108    2558 scope.go:117] "RemoveContainer" containerID="9a09904529188db77e602c980a1eb2f9d77a00c8349f98931028b731470ac74c"
Sep 18 18:41:51.611341 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:41:51.611325    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a09904529188db77e602c980a1eb2f9d77a00c8349f98931028b731470ac74c\": container with ID starting with 9a09904529188db77e602c980a1eb2f9d77a00c8349f98931028b731470ac74c not found: ID does not exist" containerID="9a09904529188db77e602c980a1eb2f9d77a00c8349f98931028b731470ac74c"
Sep 18 18:41:51.611376 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.611347    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a09904529188db77e602c980a1eb2f9d77a00c8349f98931028b731470ac74c"} err="failed to get container status \"9a09904529188db77e602c980a1eb2f9d77a00c8349f98931028b731470ac74c\": rpc error: code = NotFound desc = could not find container \"9a09904529188db77e602c980a1eb2f9d77a00c8349f98931028b731470ac74c\": container with ID starting with 9a09904529188db77e602c980a1eb2f9d77a00c8349f98931028b731470ac74c not found: ID does not exist"
Sep 18 18:41:51.611376 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.611362    2558 scope.go:117] "RemoveContainer" containerID="db401c07488a6ed602a18d8956218024c4eb4358553fa5eebca68ad7d340363b"
Sep 18 18:41:51.611602 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:41:51.611587    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db401c07488a6ed602a18d8956218024c4eb4358553fa5eebca68ad7d340363b\": container with ID starting with db401c07488a6ed602a18d8956218024c4eb4358553fa5eebca68ad7d340363b not found: ID does not exist" containerID="db401c07488a6ed602a18d8956218024c4eb4358553fa5eebca68ad7d340363b"
Sep 18 18:41:51.611634 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.611606    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db401c07488a6ed602a18d8956218024c4eb4358553fa5eebca68ad7d340363b"} err="failed to get container status \"db401c07488a6ed602a18d8956218024c4eb4358553fa5eebca68ad7d340363b\": rpc error: code = NotFound desc = could not find container \"db401c07488a6ed602a18d8956218024c4eb4358553fa5eebca68ad7d340363b\": container with ID starting with db401c07488a6ed602a18d8956218024c4eb4358553fa5eebca68ad7d340363b not found: ID does not exist"
Sep 18 18:41:51.611634 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.611619    2558 scope.go:117] "RemoveContainer" containerID="3315d42aa6ffd1c04bf7e0a7f084d655d0f29e0422b0295a858bc47cd954fad4"
Sep 18 18:41:51.611881 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:41:51.611860    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3315d42aa6ffd1c04bf7e0a7f084d655d0f29e0422b0295a858bc47cd954fad4\": container with ID starting with 3315d42aa6ffd1c04bf7e0a7f084d655d0f29e0422b0295a858bc47cd954fad4 not found: ID does not exist" containerID="3315d42aa6ffd1c04bf7e0a7f084d655d0f29e0422b0295a858bc47cd954fad4"
Sep 18 18:41:51.611971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.611885    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3315d42aa6ffd1c04bf7e0a7f084d655d0f29e0422b0295a858bc47cd954fad4"} err="failed to get container status \"3315d42aa6ffd1c04bf7e0a7f084d655d0f29e0422b0295a858bc47cd954fad4\": rpc error: code = NotFound desc = could not find container \"3315d42aa6ffd1c04bf7e0a7f084d655d0f29e0422b0295a858bc47cd954fad4\": container with ID starting with 3315d42aa6ffd1c04bf7e0a7f084d655d0f29e0422b0295a858bc47cd954fad4 not found: ID does not exist"
Sep 18 18:41:51.611971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.611900    2558 scope.go:117] "RemoveContainer" containerID="554c636d2bc34792fc5cc4723cfb078c28aa465e1a78658feb3b54dc663db3c8"
Sep 18 18:41:51.612153 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:41:51.612136    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"554c636d2bc34792fc5cc4723cfb078c28aa465e1a78658feb3b54dc663db3c8\": container with ID starting with 554c636d2bc34792fc5cc4723cfb078c28aa465e1a78658feb3b54dc663db3c8 not found: ID does not exist" containerID="554c636d2bc34792fc5cc4723cfb078c28aa465e1a78658feb3b54dc663db3c8"
Sep 18 18:41:51.612197 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.612157    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"554c636d2bc34792fc5cc4723cfb078c28aa465e1a78658feb3b54dc663db3c8"} err="failed to get container status \"554c636d2bc34792fc5cc4723cfb078c28aa465e1a78658feb3b54dc663db3c8\": rpc error: code = NotFound desc = could not find container \"554c636d2bc34792fc5cc4723cfb078c28aa465e1a78658feb3b54dc663db3c8\": container with ID starting with 554c636d2bc34792fc5cc4723cfb078c28aa465e1a78658feb3b54dc663db3c8 not found: ID does not exist"
Sep 18 18:41:51.612197 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.612169    2558 scope.go:117] "RemoveContainer" containerID="e78a47072848863b62e1a944ef54339207e014e4d0e07e8b01405ef48928af79"
Sep 18 18:41:51.612355 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:41:51.612340    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e78a47072848863b62e1a944ef54339207e014e4d0e07e8b01405ef48928af79\": container with ID starting with e78a47072848863b62e1a944ef54339207e014e4d0e07e8b01405ef48928af79 not found: ID does not exist" containerID="e78a47072848863b62e1a944ef54339207e014e4d0e07e8b01405ef48928af79"
Sep 18 18:41:51.612401 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.612358    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e78a47072848863b62e1a944ef54339207e014e4d0e07e8b01405ef48928af79"} err="failed to get container status \"e78a47072848863b62e1a944ef54339207e014e4d0e07e8b01405ef48928af79\": rpc error: code = NotFound desc = could not find container \"e78a47072848863b62e1a944ef54339207e014e4d0e07e8b01405ef48928af79\": container with ID starting with e78a47072848863b62e1a944ef54339207e014e4d0e07e8b01405ef48928af79 not found: ID does not exist"
Sep 18 18:41:51.612401 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.612371    2558 scope.go:117] "RemoveContainer" containerID="f9765782d3ae8d6eda57c80d791e53dcd3f889756156d9dc0b8d25cdff7f957f"
Sep 18 18:41:51.612610 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:41:51.612594    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9765782d3ae8d6eda57c80d791e53dcd3f889756156d9dc0b8d25cdff7f957f\": container with ID starting with f9765782d3ae8d6eda57c80d791e53dcd3f889756156d9dc0b8d25cdff7f957f not found: ID does not exist" containerID="f9765782d3ae8d6eda57c80d791e53dcd3f889756156d9dc0b8d25cdff7f957f"
Sep 18 18:41:51.612662 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.612615    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9765782d3ae8d6eda57c80d791e53dcd3f889756156d9dc0b8d25cdff7f957f"} err="failed to get container status \"f9765782d3ae8d6eda57c80d791e53dcd3f889756156d9dc0b8d25cdff7f957f\": rpc error: code = NotFound desc = could not find container \"f9765782d3ae8d6eda57c80d791e53dcd3f889756156d9dc0b8d25cdff7f957f\": container with ID starting with f9765782d3ae8d6eda57c80d791e53dcd3f889756156d9dc0b8d25cdff7f957f not found: ID does not exist"
Sep 18 18:41:51.619644 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.619622    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 18:41:51.619958 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.619941    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="kube-rbac-proxy-metric"
Sep 18 18:41:51.619958 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.619958    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="kube-rbac-proxy-metric"
Sep 18 18:41:51.620087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.619969    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="alertmanager"
Sep 18 18:41:51.620087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.619977    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="alertmanager"
Sep 18 18:41:51.620087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.619989    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="prom-label-proxy"
Sep 18 18:41:51.620087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.619998    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="prom-label-proxy"
Sep 18 18:41:51.620087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620008    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="init-config-reloader"
Sep 18 18:41:51.620087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620016    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="init-config-reloader"
Sep 18 18:41:51.620087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620030    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dfd0098-ac9b-4252-b7f6-ef613c2eaca2" containerName="console"
Sep 18 18:41:51.620087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620038    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dfd0098-ac9b-4252-b7f6-ef613c2eaca2" containerName="console"
Sep 18 18:41:51.620087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620053    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="kube-rbac-proxy-web"
Sep 18 18:41:51.620087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620063    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="kube-rbac-proxy-web"
Sep 18 18:41:51.620087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620078    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="kube-rbac-proxy"
Sep 18 18:41:51.620087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620086    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="kube-rbac-proxy"
Sep 18 18:41:51.620464 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620098    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="config-reloader"
Sep 18 18:41:51.620464 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620106    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="config-reloader"
Sep 18 18:41:51.620464 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620144    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19a8a2ad-9601-42ef-b933-6dbf04a95758" containerName="console"
Sep 18 18:41:51.620464 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620152    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="19a8a2ad-9601-42ef-b933-6dbf04a95758" containerName="console"
Sep 18 18:41:51.620464 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620236    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="alertmanager"
Sep 18 18:41:51.620464 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620248    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="kube-rbac-proxy-web"
Sep 18 18:41:51.620464 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620256    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="kube-rbac-proxy"
Sep 18 18:41:51.620464 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620265    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="config-reloader"
Sep 18 18:41:51.620464 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620274    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dfd0098-ac9b-4252-b7f6-ef613c2eaca2" containerName="console"
Sep 18 18:41:51.620464 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620282    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="prom-label-proxy"
Sep 18 18:41:51.620464 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620291    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" containerName="kube-rbac-proxy-metric"
Sep 18 18:41:51.620464 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.620300    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="19a8a2ad-9601-42ef-b933-6dbf04a95758" containerName="console"
Sep 18 18:41:51.624265 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.624248    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.630052 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.629292    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 18 18:41:51.630052 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.629546    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 18 18:41:51.630052 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.629588    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 18 18:41:51.630052 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.629870    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 18 18:41:51.630052 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.629932    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 18 18:41:51.630052 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.629948    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 18 18:41:51.630052 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.629948    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 18 18:41:51.630052 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.629873    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4bzds\""
Sep 18 18:41:51.630633 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.630616    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 18 18:41:51.633613 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.633593    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 18 18:41:51.636563 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.636539    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 18:41:51.791186 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.791150    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-web-config\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.791186 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.791191    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/567c4d29-eadb-49fc-ba4a-65f9cedabb99-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.791384 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.791219    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/567c4d29-eadb-49fc-ba4a-65f9cedabb99-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.791384 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.791262    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-config-volume\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.791384 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.791281    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.791384 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.791301    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/567c4d29-eadb-49fc-ba4a-65f9cedabb99-tls-assets\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.791508 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.791378    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/567c4d29-eadb-49fc-ba4a-65f9cedabb99-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.791508 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.791409    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.791508 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.791433    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.791508 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.791456    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/567c4d29-eadb-49fc-ba4a-65f9cedabb99-config-out\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.791508 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.791474    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.791508 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.791499    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c7tn\" (UniqueName: \"kubernetes.io/projected/567c4d29-eadb-49fc-ba4a-65f9cedabb99-kube-api-access-4c7tn\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.791677 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.791533    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.843038 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.843013    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b35f223-b9bd-4c55-95f6-b7acd9a5e63c" path="/var/lib/kubelet/pods/9b35f223-b9bd-4c55-95f6-b7acd9a5e63c/volumes"
Sep 18 18:41:51.892164 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.892143    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/567c4d29-eadb-49fc-ba4a-65f9cedabb99-tls-assets\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.892231 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.892172    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/567c4d29-eadb-49fc-ba4a-65f9cedabb99-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.892231 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.892193    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.892231 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.892216    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.892367 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.892244    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/567c4d29-eadb-49fc-ba4a-65f9cedabb99-config-out\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.892367 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.892270    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.892367 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.892299    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c7tn\" (UniqueName: \"kubernetes.io/projected/567c4d29-eadb-49fc-ba4a-65f9cedabb99-kube-api-access-4c7tn\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.892367 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.892343    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.892518 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.892380    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-web-config\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.892518 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.892415    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/567c4d29-eadb-49fc-ba4a-65f9cedabb99-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.892518 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.892439    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/567c4d29-eadb-49fc-ba4a-65f9cedabb99-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.892518 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.892481    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-config-volume\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.892518 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.892515    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.893127 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.892954    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/567c4d29-eadb-49fc-ba4a-65f9cedabb99-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.893291 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.893220    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/567c4d29-eadb-49fc-ba4a-65f9cedabb99-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.895359 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.895294    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.895359 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.895303    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/567c4d29-eadb-49fc-ba4a-65f9cedabb99-config-out\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.895359 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.895316    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.895587 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.895408    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.895587 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.895514    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.895675 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.895588    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.895803 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.895781    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-web-config\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.895932 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.895913    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/567c4d29-eadb-49fc-ba4a-65f9cedabb99-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.895991 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.895961    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/567c4d29-eadb-49fc-ba4a-65f9cedabb99-tls-assets\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.897190 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.897170    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/567c4d29-eadb-49fc-ba4a-65f9cedabb99-config-volume\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.900606 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.900581    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c7tn\" (UniqueName: \"kubernetes.io/projected/567c4d29-eadb-49fc-ba4a-65f9cedabb99-kube-api-access-4c7tn\") pod \"alertmanager-main-0\" (UID: \"567c4d29-eadb-49fc-ba4a-65f9cedabb99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:51.939417 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:51.939398    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:41:52.240044 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:52.240015    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 18:41:52.241272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:41:52.241243    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod567c4d29_eadb_49fc_ba4a_65f9cedabb99.slice/crio-235cdbf7a01213e3e011a3962ad4c69c51e88a8ea6051caf729ddd07782e6a48 WatchSource:0}: Error finding container 235cdbf7a01213e3e011a3962ad4c69c51e88a8ea6051caf729ddd07782e6a48: Status 404 returned error can't find the container with id 235cdbf7a01213e3e011a3962ad4c69c51e88a8ea6051caf729ddd07782e6a48
Sep 18 18:41:52.569174 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:52.569143    2558 generic.go:356] "Generic (PLEG): container finished" podID="567c4d29-eadb-49fc-ba4a-65f9cedabb99" containerID="c384258ce71da6066a7890ec47de61cc27966b30b3fec58371277ff29f2d0c09" exitCode=0
Sep 18 18:41:52.569323 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:52.569212    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"567c4d29-eadb-49fc-ba4a-65f9cedabb99","Type":"ContainerDied","Data":"c384258ce71da6066a7890ec47de61cc27966b30b3fec58371277ff29f2d0c09"}
Sep 18 18:41:52.569323 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:52.569264    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"567c4d29-eadb-49fc-ba4a-65f9cedabb99","Type":"ContainerStarted","Data":"235cdbf7a01213e3e011a3962ad4c69c51e88a8ea6051caf729ddd07782e6a48"}
Sep 18 18:41:53.576857 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:53.576801    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"567c4d29-eadb-49fc-ba4a-65f9cedabb99","Type":"ContainerStarted","Data":"24dddd29d24eed2e00b04da752dc21730992e27d27a17f67865de66e63840e9a"}
Sep 18 18:41:53.577208 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:53.576901    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"567c4d29-eadb-49fc-ba4a-65f9cedabb99","Type":"ContainerStarted","Data":"4a7b50e50ca562f233a26ffc66ec79044d62b295eac0c0e73f85a44c6ead5d9e"}
Sep 18 18:41:53.577208 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:53.576947    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"567c4d29-eadb-49fc-ba4a-65f9cedabb99","Type":"ContainerStarted","Data":"1b73d25a6b8ac5b657deab4ba5b526125814bb2f3ab3e9cfa1721d55e6895d36"}
Sep 18 18:41:53.577208 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:53.576983    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"567c4d29-eadb-49fc-ba4a-65f9cedabb99","Type":"ContainerStarted","Data":"74369b04d66276a753e18b9221b7a350451e2520140374089f9c58c8746bd4e9"}
Sep 18 18:41:53.577208 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:53.577018    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"567c4d29-eadb-49fc-ba4a-65f9cedabb99","Type":"ContainerStarted","Data":"8ce025602690004d5743cdb8581cd5725d927f2486543d80a8354e9f656bf8a7"}
Sep 18 18:41:53.577208 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:53.577057    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"567c4d29-eadb-49fc-ba4a-65f9cedabb99","Type":"ContainerStarted","Data":"7957eadd96c9ecc622ba90cd7c1febf136d02d9e77756ec95da047a85f4009a2"}
Sep 18 18:41:53.607330 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:53.607276    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.591774838 podStartE2EDuration="2.607260422s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.485584ms" isStatefulPod=false podCreationTimestamp="2026-09-18 18:41:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:41:53.604739551 +0000 UTC m=+149.803223787" watchObservedRunningTime="2026-09-18 18:41:53.607260422 +0000 UTC m=+149.805744660"
Sep 18 18:41:54.074779 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.074742    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 18:41:54.075114 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.075083    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="kube-rbac-proxy" containerID="cri-o://9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac" gracePeriod=600
Sep 18 18:41:54.075114 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.075082    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="prometheus" containerID="cri-o://b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e" gracePeriod=600
Sep 18 18:41:54.075316 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.075121    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="thanos-sidecar" containerID="cri-o://1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae" gracePeriod=600
Sep 18 18:41:54.075316 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.075152    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="config-reloader" containerID="cri-o://edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f" gracePeriod=600
Sep 18 18:41:54.075316 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.075195    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="kube-rbac-proxy-thanos" containerID="cri-o://c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b" gracePeriod=600
Sep 18 18:41:54.075482 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.075309    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="kube-rbac-proxy-web" containerID="cri-o://377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77" gracePeriod=600
Sep 18 18:41:54.318663 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.318634    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.415609 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.415509    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/577cb189-abd2-4437-a291-85010327e8a6-config-out\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.415609 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.415550    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.415609 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.415572    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-rulefiles-1\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.415609 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.415592    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-trusted-ca-bundle\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.415957 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.415614    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-rulefiles-2\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.415957 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.415637    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-grpc-tls\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.415957 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.415676    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dppx8\" (UniqueName: \"kubernetes.io/projected/577cb189-abd2-4437-a291-85010327e8a6-kube-api-access-dppx8\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.415957 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.415704    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-thanos-prometheus-http-client-file\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.415957 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.415735    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/577cb189-abd2-4437-a291-85010327e8a6-tls-assets\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.415957 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.415759    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-config\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.415957 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.415805    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-rulefiles-0\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.415957 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.415862    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-configmap-serving-certs-ca-bundle\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.415957 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.415887    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:41:54.415957 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.415957    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-metrics-client-certs\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.416413 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.415971    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:41:54.416413 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.415990    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.416413 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.416032    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-kube-rbac-proxy\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.416413 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.416055    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-configmap-metrics-client-ca\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.416413 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.416088    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-configmap-kubelet-serving-ca-bundle\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.416413 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.416099    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:41:54.416413 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.416154    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-web-config\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.416413 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.416192    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-prometheus-k8s-tls\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.416413 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.416219    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-db\") pod \"577cb189-abd2-4437-a291-85010327e8a6\" (UID: \"577cb189-abd2-4437-a291-85010327e8a6\") "
Sep 18 18:41:54.416850 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.416507    2558 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.416850 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.416527    2558 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.416850 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.416544    2558 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.417059 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.416860    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:41:54.417110 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.417085    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:41:54.417446 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.417298    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:41:54.417650 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.417625    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:41:54.418928 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.418897    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:41:54.419599 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.419557    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:41:54.419797 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.419761    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:41:54.420399 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.420377    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-config" (OuterVolumeSpecName: "config") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:41:54.420493 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.420396    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/577cb189-abd2-4437-a291-85010327e8a6-kube-api-access-dppx8" (OuterVolumeSpecName: "kube-api-access-dppx8") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "kube-api-access-dppx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:41:54.420552 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.420527    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:41:54.420780 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.420752    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/577cb189-abd2-4437-a291-85010327e8a6-config-out" (OuterVolumeSpecName: "config-out") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:41:54.420899 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.420792    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:41:54.420899 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.420885    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:41:54.420994 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.420942    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:41:54.421531 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.421509    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/577cb189-abd2-4437-a291-85010327e8a6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:41:54.422396 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.422380    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:41:54.431620 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.431599    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-web-config" (OuterVolumeSpecName: "web-config") pod "577cb189-abd2-4437-a291-85010327e8a6" (UID: "577cb189-abd2-4437-a291-85010327e8a6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:41:54.517534 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.517489    2558 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-grpc-tls\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.517534 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.517528    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dppx8\" (UniqueName: \"kubernetes.io/projected/577cb189-abd2-4437-a291-85010327e8a6-kube-api-access-dppx8\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.517534 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.517540    2558 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.517534 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.517550    2558 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/577cb189-abd2-4437-a291-85010327e8a6-tls-assets\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.517771 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.517559    2558 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-config\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.517771 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.517566    2558 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.517771 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.517575    2558 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.517771 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.517585    2558 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-metrics-client-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.517771 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.517594    2558 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.517771 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.517603    2558 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-kube-rbac-proxy\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.517771 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.517611    2558 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-configmap-metrics-client-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.517771 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.517619    2558 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/577cb189-abd2-4437-a291-85010327e8a6-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.517771 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.517628    2558 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-web-config\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.517771 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.517635    2558 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.517771 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.517644    2558 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/577cb189-abd2-4437-a291-85010327e8a6-prometheus-k8s-db\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.517771 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.517651    2558 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/577cb189-abd2-4437-a291-85010327e8a6-config-out\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.517771 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.517659    2558 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/577cb189-abd2-4437-a291-85010327e8a6-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:41:54.584787 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.584754    2558 generic.go:356] "Generic (PLEG): container finished" podID="577cb189-abd2-4437-a291-85010327e8a6" containerID="c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b" exitCode=0
Sep 18 18:41:54.584787 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.584786    2558 generic.go:356] "Generic (PLEG): container finished" podID="577cb189-abd2-4437-a291-85010327e8a6" containerID="9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac" exitCode=0
Sep 18 18:41:54.585173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.584796    2558 generic.go:356] "Generic (PLEG): container finished" podID="577cb189-abd2-4437-a291-85010327e8a6" containerID="377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77" exitCode=0
Sep 18 18:41:54.585173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.584805    2558 generic.go:356] "Generic (PLEG): container finished" podID="577cb189-abd2-4437-a291-85010327e8a6" containerID="1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae" exitCode=0
Sep 18 18:41:54.585173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.584812    2558 generic.go:356] "Generic (PLEG): container finished" podID="577cb189-abd2-4437-a291-85010327e8a6" containerID="edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f" exitCode=0
Sep 18 18:41:54.585173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.584843    2558 generic.go:356] "Generic (PLEG): container finished" podID="577cb189-abd2-4437-a291-85010327e8a6" containerID="b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e" exitCode=0
Sep 18 18:41:54.585173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.584855    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"577cb189-abd2-4437-a291-85010327e8a6","Type":"ContainerDied","Data":"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b"}
Sep 18 18:41:54.585173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.584919    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"577cb189-abd2-4437-a291-85010327e8a6","Type":"ContainerDied","Data":"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac"}
Sep 18 18:41:54.585173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.584866    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.585173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.584946    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"577cb189-abd2-4437-a291-85010327e8a6","Type":"ContainerDied","Data":"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77"}
Sep 18 18:41:54.585173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.584974    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"577cb189-abd2-4437-a291-85010327e8a6","Type":"ContainerDied","Data":"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae"}
Sep 18 18:41:54.585173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.585000    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"577cb189-abd2-4437-a291-85010327e8a6","Type":"ContainerDied","Data":"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f"}
Sep 18 18:41:54.585173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.585022    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"577cb189-abd2-4437-a291-85010327e8a6","Type":"ContainerDied","Data":"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e"}
Sep 18 18:41:54.585173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.585022    2558 scope.go:117] "RemoveContainer" containerID="c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b"
Sep 18 18:41:54.585173 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.585047    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"577cb189-abd2-4437-a291-85010327e8a6","Type":"ContainerDied","Data":"b36d0c3852190f80959c07ec40053acc0346e122c7b7e45fbccd8fb2e06ca445"}
Sep 18 18:41:54.593755 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.593737    2558 scope.go:117] "RemoveContainer" containerID="9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac"
Sep 18 18:41:54.600654 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.600640    2558 scope.go:117] "RemoveContainer" containerID="377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77"
Sep 18 18:41:54.607155 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.607141    2558 scope.go:117] "RemoveContainer" containerID="1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae"
Sep 18 18:41:54.610409 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.610389    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 18:41:54.614296 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.614278    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 18:41:54.614839 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.614808    2558 scope.go:117] "RemoveContainer" containerID="edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f"
Sep 18 18:41:54.621399 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.621383    2558 scope.go:117] "RemoveContainer" containerID="b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e"
Sep 18 18:41:54.628101 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.628087    2558 scope.go:117] "RemoveContainer" containerID="ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed"
Sep 18 18:41:54.634736 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.634722    2558 scope.go:117] "RemoveContainer" containerID="c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b"
Sep 18 18:41:54.635009 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:41:54.634992    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b\": container with ID starting with c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b not found: ID does not exist" containerID="c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b"
Sep 18 18:41:54.635062 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.635018    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b"} err="failed to get container status \"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b\": rpc error: code = NotFound desc = could not find container \"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b\": container with ID starting with c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b not found: ID does not exist"
Sep 18 18:41:54.635062 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.635037    2558 scope.go:117] "RemoveContainer" containerID="9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac"
Sep 18 18:41:54.635313 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:41:54.635295    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac\": container with ID starting with 9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac not found: ID does not exist" containerID="9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac"
Sep 18 18:41:54.635427 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.635322    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac"} err="failed to get container status \"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac\": rpc error: code = NotFound desc = could not find container \"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac\": container with ID starting with 9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac not found: ID does not exist"
Sep 18 18:41:54.635427 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.635344    2558 scope.go:117] "RemoveContainer" containerID="377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77"
Sep 18 18:41:54.635607 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:41:54.635588    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77\": container with ID starting with 377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77 not found: ID does not exist" containerID="377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77"
Sep 18 18:41:54.635646 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.635615    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77"} err="failed to get container status \"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77\": rpc error: code = NotFound desc = could not find container \"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77\": container with ID starting with 377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77 not found: ID does not exist"
Sep 18 18:41:54.635646 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.635636    2558 scope.go:117] "RemoveContainer" containerID="1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae"
Sep 18 18:41:54.635886 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:41:54.635868    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae\": container with ID starting with 1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae not found: ID does not exist" containerID="1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae"
Sep 18 18:41:54.635969 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.635893    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae"} err="failed to get container status \"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae\": rpc error: code = NotFound desc = could not find container \"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae\": container with ID starting with 1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae not found: ID does not exist"
Sep 18 18:41:54.635969 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.635915    2558 scope.go:117] "RemoveContainer" containerID="edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f"
Sep 18 18:41:54.636151 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:41:54.636136    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f\": container with ID starting with edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f not found: ID does not exist" containerID="edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f"
Sep 18 18:41:54.636198 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.636156    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f"} err="failed to get container status \"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f\": rpc error: code = NotFound desc = could not find container \"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f\": container with ID starting with edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f not found: ID does not exist"
Sep 18 18:41:54.636198 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.636169    2558 scope.go:117] "RemoveContainer" containerID="b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e"
Sep 18 18:41:54.636411 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:41:54.636393    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e\": container with ID starting with b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e not found: ID does not exist" containerID="b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e"
Sep 18 18:41:54.636455 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.636417    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e"} err="failed to get container status \"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e\": rpc error: code = NotFound desc = could not find container \"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e\": container with ID starting with b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e not found: ID does not exist"
Sep 18 18:41:54.636455 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.636436    2558 scope.go:117] "RemoveContainer" containerID="ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed"
Sep 18 18:41:54.636668 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:41:54.636651    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed\": container with ID starting with ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed not found: ID does not exist" containerID="ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed"
Sep 18 18:41:54.636731 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.636675    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed"} err="failed to get container status \"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed\": rpc error: code = NotFound desc = could not find container \"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed\": container with ID starting with ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed not found: ID does not exist"
Sep 18 18:41:54.636731 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.636692    2558 scope.go:117] "RemoveContainer" containerID="c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b"
Sep 18 18:41:54.636939 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.636921    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b"} err="failed to get container status \"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b\": rpc error: code = NotFound desc = could not find container \"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b\": container with ID starting with c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b not found: ID does not exist"
Sep 18 18:41:54.636998 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.636940    2558 scope.go:117] "RemoveContainer" containerID="9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac"
Sep 18 18:41:54.637146 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.637128    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac"} err="failed to get container status \"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac\": rpc error: code = NotFound desc = could not find container \"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac\": container with ID starting with 9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac not found: ID does not exist"
Sep 18 18:41:54.637219 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.637148    2558 scope.go:117] "RemoveContainer" containerID="377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77"
Sep 18 18:41:54.637404 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.637387    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77"} err="failed to get container status \"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77\": rpc error: code = NotFound desc = could not find container \"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77\": container with ID starting with 377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77 not found: ID does not exist"
Sep 18 18:41:54.637441 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.637404    2558 scope.go:117] "RemoveContainer" containerID="1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae"
Sep 18 18:41:54.637609 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.637591    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae"} err="failed to get container status \"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae\": rpc error: code = NotFound desc = could not find container \"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae\": container with ID starting with 1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae not found: ID does not exist"
Sep 18 18:41:54.637655 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.637611    2558 scope.go:117] "RemoveContainer" containerID="edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f"
Sep 18 18:41:54.637813 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.637795    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f"} err="failed to get container status \"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f\": rpc error: code = NotFound desc = could not find container \"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f\": container with ID starting with edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f not found: ID does not exist"
Sep 18 18:41:54.637813 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.637813    2558 scope.go:117] "RemoveContainer" containerID="b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e"
Sep 18 18:41:54.638047 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.638027    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e"} err="failed to get container status \"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e\": rpc error: code = NotFound desc = could not find container \"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e\": container with ID starting with b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e not found: ID does not exist"
Sep 18 18:41:54.638085 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.638049    2558 scope.go:117] "RemoveContainer" containerID="ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed"
Sep 18 18:41:54.638364 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.638316    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed"} err="failed to get container status \"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed\": rpc error: code = NotFound desc = could not find container \"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed\": container with ID starting with ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed not found: ID does not exist"
Sep 18 18:41:54.638364 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.638340    2558 scope.go:117] "RemoveContainer" containerID="c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b"
Sep 18 18:41:54.638723 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.638695    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b"} err="failed to get container status \"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b\": rpc error: code = NotFound desc = could not find container \"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b\": container with ID starting with c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b not found: ID does not exist"
Sep 18 18:41:54.638723 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.638722    2558 scope.go:117] "RemoveContainer" containerID="9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac"
Sep 18 18:41:54.639021 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.638987    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac"} err="failed to get container status \"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac\": rpc error: code = NotFound desc = could not find container \"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac\": container with ID starting with 9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac not found: ID does not exist"
Sep 18 18:41:54.639101 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.639022    2558 scope.go:117] "RemoveContainer" containerID="377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77"
Sep 18 18:41:54.639338 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.639312    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77"} err="failed to get container status \"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77\": rpc error: code = NotFound desc = could not find container \"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77\": container with ID starting with 377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77 not found: ID does not exist"
Sep 18 18:41:54.639434 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.639338    2558 scope.go:117] "RemoveContainer" containerID="1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae"
Sep 18 18:41:54.639634 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.639612    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae"} err="failed to get container status \"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae\": rpc error: code = NotFound desc = could not find container \"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae\": container with ID starting with 1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae not found: ID does not exist"
Sep 18 18:41:54.639695 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.639636    2558 scope.go:117] "RemoveContainer" containerID="edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f"
Sep 18 18:41:54.639908 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.639889    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f"} err="failed to get container status \"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f\": rpc error: code = NotFound desc = could not find container \"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f\": container with ID starting with edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f not found: ID does not exist"
Sep 18 18:41:54.639957 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.639910    2558 scope.go:117] "RemoveContainer" containerID="b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e"
Sep 18 18:41:54.640159 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640139    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e"} err="failed to get container status \"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e\": rpc error: code = NotFound desc = could not find container \"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e\": container with ID starting with b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e not found: ID does not exist"
Sep 18 18:41:54.640159 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640157    2558 scope.go:117] "RemoveContainer" containerID="ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed"
Sep 18 18:41:54.640439 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640424    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed"} err="failed to get container status \"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed\": rpc error: code = NotFound desc = could not find container \"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed\": container with ID starting with ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed not found: ID does not exist"
Sep 18 18:41:54.640439 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640438    2558 scope.go:117] "RemoveContainer" containerID="c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b"
Sep 18 18:41:54.640539 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640523    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 18:41:54.640675 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640658    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b"} err="failed to get container status \"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b\": rpc error: code = NotFound desc = could not find container \"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b\": container with ID starting with c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b not found: ID does not exist"
Sep 18 18:41:54.640712 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640676    2558 scope.go:117] "RemoveContainer" containerID="9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac"
Sep 18 18:41:54.640856 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640843    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="thanos-sidecar"
Sep 18 18:41:54.640856 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640856    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="thanos-sidecar"
Sep 18 18:41:54.640937 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640866    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="config-reloader"
Sep 18 18:41:54.640937 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640874    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="config-reloader"
Sep 18 18:41:54.640937 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640887    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="init-config-reloader"
Sep 18 18:41:54.640937 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640893    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="init-config-reloader"
Sep 18 18:41:54.640937 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640892    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac"} err="failed to get container status \"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac\": rpc error: code = NotFound desc = could not find container \"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac\": container with ID starting with 9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac not found: ID does not exist"
Sep 18 18:41:54.640937 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640903    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="kube-rbac-proxy-web"
Sep 18 18:41:54.640937 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640906    2558 scope.go:117] "RemoveContainer" containerID="377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77"
Sep 18 18:41:54.641167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640909    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="kube-rbac-proxy-web"
Sep 18 18:41:54.641167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640956    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="kube-rbac-proxy"
Sep 18 18:41:54.641167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640963    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="kube-rbac-proxy"
Sep 18 18:41:54.641167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640974    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="kube-rbac-proxy-thanos"
Sep 18 18:41:54.641167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640979    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="kube-rbac-proxy-thanos"
Sep 18 18:41:54.641167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640992    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="prometheus"
Sep 18 18:41:54.641167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.640998    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="prometheus"
Sep 18 18:41:54.641167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.641078    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="kube-rbac-proxy"
Sep 18 18:41:54.641167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.641084    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="kube-rbac-proxy-thanos"
Sep 18 18:41:54.641167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.641092    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="prometheus"
Sep 18 18:41:54.641167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.641098    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="kube-rbac-proxy-web"
Sep 18 18:41:54.641167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.641104    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="config-reloader"
Sep 18 18:41:54.641167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.641109    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="577cb189-abd2-4437-a291-85010327e8a6" containerName="thanos-sidecar"
Sep 18 18:41:54.641167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.641158    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77"} err="failed to get container status \"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77\": rpc error: code = NotFound desc = could not find container \"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77\": container with ID starting with 377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77 not found: ID does not exist"
Sep 18 18:41:54.641167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.641172    2558 scope.go:117] "RemoveContainer" containerID="1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae"
Sep 18 18:41:54.641713 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.641362    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae"} err="failed to get container status \"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae\": rpc error: code = NotFound desc = could not find container \"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae\": container with ID starting with 1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae not found: ID does not exist"
Sep 18 18:41:54.641713 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.641374    2558 scope.go:117] "RemoveContainer" containerID="edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f"
Sep 18 18:41:54.641713 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.641657    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f"} err="failed to get container status \"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f\": rpc error: code = NotFound desc = could not find container \"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f\": container with ID starting with edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f not found: ID does not exist"
Sep 18 18:41:54.641713 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.641678    2558 scope.go:117] "RemoveContainer" containerID="b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e"
Sep 18 18:41:54.642264 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.642147    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e"} err="failed to get container status \"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e\": rpc error: code = NotFound desc = could not find container \"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e\": container with ID starting with b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e not found: ID does not exist"
Sep 18 18:41:54.642264 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.642171    2558 scope.go:117] "RemoveContainer" containerID="ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed"
Sep 18 18:41:54.642471 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.642445    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed"} err="failed to get container status \"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed\": rpc error: code = NotFound desc = could not find container \"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed\": container with ID starting with ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed not found: ID does not exist"
Sep 18 18:41:54.642546 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.642474    2558 scope.go:117] "RemoveContainer" containerID="c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b"
Sep 18 18:41:54.642768 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.642743    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b"} err="failed to get container status \"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b\": rpc error: code = NotFound desc = could not find container \"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b\": container with ID starting with c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b not found: ID does not exist"
Sep 18 18:41:54.642768 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.642766    2558 scope.go:117] "RemoveContainer" containerID="9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac"
Sep 18 18:41:54.643015 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.642999    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac"} err="failed to get container status \"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac\": rpc error: code = NotFound desc = could not find container \"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac\": container with ID starting with 9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac not found: ID does not exist"
Sep 18 18:41:54.643015 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.643015    2558 scope.go:117] "RemoveContainer" containerID="377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77"
Sep 18 18:41:54.643264 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.643246    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77"} err="failed to get container status \"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77\": rpc error: code = NotFound desc = could not find container \"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77\": container with ID starting with 377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77 not found: ID does not exist"
Sep 18 18:41:54.643327 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.643265    2558 scope.go:117] "RemoveContainer" containerID="1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae"
Sep 18 18:41:54.643576 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.643540    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae"} err="failed to get container status \"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae\": rpc error: code = NotFound desc = could not find container \"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae\": container with ID starting with 1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae not found: ID does not exist"
Sep 18 18:41:54.643576 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.643571    2558 scope.go:117] "RemoveContainer" containerID="edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f"
Sep 18 18:41:54.643855 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.643831    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f"} err="failed to get container status \"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f\": rpc error: code = NotFound desc = could not find container \"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f\": container with ID starting with edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f not found: ID does not exist"
Sep 18 18:41:54.643855 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.643855    2558 scope.go:117] "RemoveContainer" containerID="b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e"
Sep 18 18:41:54.644094 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.644078    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e"} err="failed to get container status \"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e\": rpc error: code = NotFound desc = could not find container \"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e\": container with ID starting with b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e not found: ID does not exist"
Sep 18 18:41:54.644142 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.644094    2558 scope.go:117] "RemoveContainer" containerID="ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed"
Sep 18 18:41:54.644293 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.644277    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed"} err="failed to get container status \"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed\": rpc error: code = NotFound desc = could not find container \"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed\": container with ID starting with ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed not found: ID does not exist"
Sep 18 18:41:54.644330 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.644293    2558 scope.go:117] "RemoveContainer" containerID="c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b"
Sep 18 18:41:54.644499 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.644484    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b"} err="failed to get container status \"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b\": rpc error: code = NotFound desc = could not find container \"c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b\": container with ID starting with c23d97af649aaaea35e2615ecc925925e79f908599ca065e4a95a5aca9afc02b not found: ID does not exist"
Sep 18 18:41:54.644548 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.644498    2558 scope.go:117] "RemoveContainer" containerID="9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac"
Sep 18 18:41:54.644688 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.644670    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac"} err="failed to get container status \"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac\": rpc error: code = NotFound desc = could not find container \"9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac\": container with ID starting with 9985fed705b1bc7ec201068b120450c7ac560a5feadb37ed61f37d4b7f4667ac not found: ID does not exist"
Sep 18 18:41:54.644736 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.644689    2558 scope.go:117] "RemoveContainer" containerID="377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77"
Sep 18 18:41:54.644889 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.644872    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77"} err="failed to get container status \"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77\": rpc error: code = NotFound desc = could not find container \"377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77\": container with ID starting with 377f4a02d4ac5ee2e06b507754a53ee578a8ed38d1a0fc11ba6d687d53578a77 not found: ID does not exist"
Sep 18 18:41:54.644929 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.644889    2558 scope.go:117] "RemoveContainer" containerID="1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae"
Sep 18 18:41:54.645118 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.645098    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae"} err="failed to get container status \"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae\": rpc error: code = NotFound desc = could not find container \"1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae\": container with ID starting with 1645f7a1fc01436645005017998ad8ab8329276820ee28cec95256d47a52b8ae not found: ID does not exist"
Sep 18 18:41:54.645169 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.645119    2558 scope.go:117] "RemoveContainer" containerID="edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f"
Sep 18 18:41:54.645331 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.645316    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f"} err="failed to get container status \"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f\": rpc error: code = NotFound desc = could not find container \"edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f\": container with ID starting with edb948893a975278c0e7a4dd9b961d685b48090c10988a5ddb7a26ab3430783f not found: ID does not exist"
Sep 18 18:41:54.645370 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.645331    2558 scope.go:117] "RemoveContainer" containerID="b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e"
Sep 18 18:41:54.645551 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.645533    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e"} err="failed to get container status \"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e\": rpc error: code = NotFound desc = could not find container \"b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e\": container with ID starting with b589e553610d38360950531763add37d52810d590e521aa9a2aaf0f83724754e not found: ID does not exist"
Sep 18 18:41:54.645619 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.645551    2558 scope.go:117] "RemoveContainer" containerID="ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed"
Sep 18 18:41:54.645669 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.645617    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.645996 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.645812    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed"} err="failed to get container status \"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed\": rpc error: code = NotFound desc = could not find container \"ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed\": container with ID starting with ec1a40d9587069d0205252d6d0722fec171f2a806f0b8d3d282f8b157b1445ed not found: ID does not exist"
Sep 18 18:41:54.649153 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.648149    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 18 18:41:54.649153 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.648555    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-73u9f6iq2gpp9\""
Sep 18 18:41:54.649153 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.648749    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 18 18:41:54.649350 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.649186    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 18 18:41:54.649350 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.649305    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 18 18:41:54.649429 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.649386    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 18 18:41:54.649429 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.649390    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 18 18:41:54.649565 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.649548    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 18 18:41:54.649658 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.649614    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 18 18:41:54.649658 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.649623    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-859rn\""
Sep 18 18:41:54.649773 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.649673    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 18 18:41:54.649773 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.649682    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 18 18:41:54.650047 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.650029    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 18 18:41:54.650365 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.650350    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 18 18:41:54.651912 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.651896    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 18 18:41:54.655071 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.655053    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 18 18:41:54.660545 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.660523    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 18:41:54.820648 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.820610    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0ed42c56-fa85-4e76-9381-c22373a94741-config-out\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.820814 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.820666    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.820814 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.820694    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.820814 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.820714    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.820814 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.820730    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0ed42c56-fa85-4e76-9381-c22373a94741-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.820814 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.820747    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.821070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.820812    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.821070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.820870    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.821070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.820888    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-config\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.821070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.820910    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.821070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.820943    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.821070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.820979    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-web-config\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.821070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.821024    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.821070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.821045    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.821070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.821069    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0ed42c56-fa85-4e76-9381-c22373a94741-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.821342 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.821085    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.821342 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.821112    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.821342 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.821133    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68jqc\" (UniqueName: \"kubernetes.io/projected/0ed42c56-fa85-4e76-9381-c22373a94741-kube-api-access-68jqc\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.821342 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.821175    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.821342 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.821196    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.922099 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922064    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.922099 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922101    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.922295 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922122    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0ed42c56-fa85-4e76-9381-c22373a94741-config-out\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.922295 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922149    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.922295 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922172    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.922295 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922196    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.922295 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922220    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0ed42c56-fa85-4e76-9381-c22373a94741-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.922295 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922244    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.922707 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922681    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.922807 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922728    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.922807 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922759    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.922807 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922766    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.922807 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922783    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-config\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.923053 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922847    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.923053 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922881    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.923053 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922909    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-web-config\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.923053 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922948    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.923053 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922952    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0ed42c56-fa85-4e76-9381-c22373a94741-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.923053 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.922978    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.923053 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.923018    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0ed42c56-fa85-4e76-9381-c22373a94741-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.923053 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.923042    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.923452 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.923067    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.923452 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.923094    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68jqc\" (UniqueName: \"kubernetes.io/projected/0ed42c56-fa85-4e76-9381-c22373a94741-kube-api-access-68jqc\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.923452 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.923394    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.923452 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.923426    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.925575 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.925457    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0ed42c56-fa85-4e76-9381-c22373a94741-config-out\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.925575 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.925540    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.925787 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.925619    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.925787 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.925636    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.925985 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.925958    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.926283 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.926258    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.926453 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.926430    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.926592 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.926573    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.927237 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.927210    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.927362 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.927339    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0ed42c56-fa85-4e76-9381-c22373a94741-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.927643 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.927621    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-config\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.927937 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.927919    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.928252 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.928235    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0ed42c56-fa85-4e76-9381-c22373a94741-web-config\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.928402 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.928385    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0ed42c56-fa85-4e76-9381-c22373a94741-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.932951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.932935    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68jqc\" (UniqueName: \"kubernetes.io/projected/0ed42c56-fa85-4e76-9381-c22373a94741-kube-api-access-68jqc\") pod \"prometheus-k8s-0\" (UID: \"0ed42c56-fa85-4e76-9381-c22373a94741\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:54.956985 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:54.956966    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:41:55.238546 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:55.238522    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 18:41:55.240436 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:41:55.240405    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ed42c56_fa85_4e76_9381_c22373a94741.slice/crio-65280cc2bd078f08e94c1820aa5dea53045214b293ed30643de2428ecf155b8b WatchSource:0}: Error finding container 65280cc2bd078f08e94c1820aa5dea53045214b293ed30643de2428ecf155b8b: Status 404 returned error can't find the container with id 65280cc2bd078f08e94c1820aa5dea53045214b293ed30643de2428ecf155b8b
Sep 18 18:41:55.592338 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:55.592298    2558 generic.go:356] "Generic (PLEG): container finished" podID="0ed42c56-fa85-4e76-9381-c22373a94741" containerID="10e317c9756fc58554ae3a2da021c87868a260e759e5708acd29a25ab5e73158" exitCode=0
Sep 18 18:41:55.592718 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:55.592387    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0ed42c56-fa85-4e76-9381-c22373a94741","Type":"ContainerDied","Data":"10e317c9756fc58554ae3a2da021c87868a260e759e5708acd29a25ab5e73158"}
Sep 18 18:41:55.592718 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:55.592481    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0ed42c56-fa85-4e76-9381-c22373a94741","Type":"ContainerStarted","Data":"65280cc2bd078f08e94c1820aa5dea53045214b293ed30643de2428ecf155b8b"}
Sep 18 18:41:55.845683 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:55.845656    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="577cb189-abd2-4437-a291-85010327e8a6" path="/var/lib/kubelet/pods/577cb189-abd2-4437-a291-85010327e8a6/volumes"
Sep 18 18:41:56.601172 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:56.601134    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0ed42c56-fa85-4e76-9381-c22373a94741","Type":"ContainerStarted","Data":"3c4cb2c6c1a198d6057250ebe3512737f84028ff4e6ac02956b4ffad213284c2"}
Sep 18 18:41:56.601616 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:56.601216    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0ed42c56-fa85-4e76-9381-c22373a94741","Type":"ContainerStarted","Data":"7870df43be05de3ae594ff29121f59649637598a3bce732cc5f8a2d661d57882"}
Sep 18 18:41:56.601616 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:56.601254    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0ed42c56-fa85-4e76-9381-c22373a94741","Type":"ContainerStarted","Data":"6503153f63baa0d743dab2517fddf625d7314196de56d4084811c3c58c49892c"}
Sep 18 18:41:56.601616 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:56.601300    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0ed42c56-fa85-4e76-9381-c22373a94741","Type":"ContainerStarted","Data":"188f40ff1bc0cd09416c3e29769a768ef01f71c7756b177001ecfc46093c596c"}
Sep 18 18:41:56.601616 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:56.601342    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0ed42c56-fa85-4e76-9381-c22373a94741","Type":"ContainerStarted","Data":"8030a6615321634ce46174528baa741d8780d72241409e0f39c73eb14d6148ab"}
Sep 18 18:41:56.601616 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:56.601380    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0ed42c56-fa85-4e76-9381-c22373a94741","Type":"ContainerStarted","Data":"7f03fbb17437fad7d719d6254fe0a4a402756a28b2641ce77b405e4d230a7179"}
Sep 18 18:41:56.642404 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:56.642348    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.62410765 podStartE2EDuration="2.642330349s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.222699ms" isStatefulPod=false podCreationTimestamp="2026-09-18 18:41:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:41:56.63086067 +0000 UTC m=+152.829344895" watchObservedRunningTime="2026-09-18 18:41:56.642330349 +0000 UTC m=+152.840814552"
Sep 18 18:41:58.522716 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.522682    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-566d77f5c7-569tf"]
Sep 18 18:41:58.524884 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.524868    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.535948 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.535919    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-566d77f5c7-569tf"]
Sep 18 18:41:58.589897 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.589838    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:41:58.655432 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.655395    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:41:58.658502 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.658480    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-trusted-ca-bundle\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.658594 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.658510    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-console-config\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.658594 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.658529    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-service-ca\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.658682 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.658618    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4ef4bbee-1cc0-4f13-871b-64edcc003c16-console-serving-cert\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.658682 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.658636    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fpjx\" (UniqueName: \"kubernetes.io/projected/4ef4bbee-1cc0-4f13-871b-64edcc003c16-kube-api-access-4fpjx\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.658682 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.658665    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4ef4bbee-1cc0-4f13-871b-64edcc003c16-console-oauth-config\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.658787 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.658733    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-oauth-serving-cert\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.760175 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.760141    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4ef4bbee-1cc0-4f13-871b-64edcc003c16-console-serving-cert\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.760340 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.760179    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fpjx\" (UniqueName: \"kubernetes.io/projected/4ef4bbee-1cc0-4f13-871b-64edcc003c16-kube-api-access-4fpjx\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.760340 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.760284    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4ef4bbee-1cc0-4f13-871b-64edcc003c16-console-oauth-config\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.760340 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.760333    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-oauth-serving-cert\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.760509 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.760375    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-trusted-ca-bundle\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.760509 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.760393    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-console-config\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.760509 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.760412    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-service-ca\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.761129 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.761106    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-console-config\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.761255 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.761159    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-oauth-serving-cert\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.761316 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.761291    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-trusted-ca-bundle\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.761591 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.761573    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-service-ca\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.763087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.763069    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4ef4bbee-1cc0-4f13-871b-64edcc003c16-console-oauth-config\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.763310 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.763292    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4ef4bbee-1cc0-4f13-871b-64edcc003c16-console-serving-cert\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.769430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.769402    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fpjx\" (UniqueName: \"kubernetes.io/projected/4ef4bbee-1cc0-4f13-871b-64edcc003c16-kube-api-access-4fpjx\") pod \"console-566d77f5c7-569tf\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") " pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:58.834976 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:58.834909    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:41:59.107311 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:59.107190    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-566d77f5c7-569tf"]
Sep 18 18:41:59.109971 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:41:59.109944    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ef4bbee_1cc0_4f13_871b_64edcc003c16.slice/crio-00b8243e0a4a4acba070530aa967b2c23c5155c54314ae801ade6e9672c940aa WatchSource:0}: Error finding container 00b8243e0a4a4acba070530aa967b2c23c5155c54314ae801ade6e9672c940aa: Status 404 returned error can't find the container with id 00b8243e0a4a4acba070530aa967b2c23c5155c54314ae801ade6e9672c940aa
Sep 18 18:41:59.614885 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:59.614849    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-566d77f5c7-569tf" event={"ID":"4ef4bbee-1cc0-4f13-871b-64edcc003c16","Type":"ContainerStarted","Data":"681dd0dc44e43bdff5cccb50f7b966589e738bf48a405b684e2afb7228b45a7e"}
Sep 18 18:41:59.615305 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:59.614909    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-566d77f5c7-569tf" event={"ID":"4ef4bbee-1cc0-4f13-871b-64edcc003c16","Type":"ContainerStarted","Data":"00b8243e0a4a4acba070530aa967b2c23c5155c54314ae801ade6e9672c940aa"}
Sep 18 18:41:59.636548 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:59.636502    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-566d77f5c7-569tf" podStartSLOduration=1.636487354 podStartE2EDuration="1.636487354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:41:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:41:59.634960565 +0000 UTC m=+155.833444772" watchObservedRunningTime="2026-09-18 18:41:59.636487354 +0000 UTC m=+155.834971585"
Sep 18 18:41:59.957386 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:41:59.957289    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:42:08.835872 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:08.835768    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:42:08.835872 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:08.835853    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:42:08.840522 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:08.840502    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:42:09.663671 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:09.663646    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:42:09.708232 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:09.708197    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c9cccb589-l6j8t"]
Sep 18 18:42:09.815924 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:09.815898    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:42:09.917334 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:09.917249    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:42:09.970565 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:09.970535    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:42:34.727656 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:34.727585    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c9cccb589-l6j8t" podUID="0e75115a-7692-43e9-90d3-438a950b71ce" containerName="console" containerID="cri-o://0bc44bff02ccc4a4a6cf68ed49fbae6e5ab7abc7c51f6457a0addf20dac3b82f" gracePeriod=15
Sep 18 18:42:34.970592 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:34.970567    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:42:35.088672 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.088646    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-service-ca\") pod \"0e75115a-7692-43e9-90d3-438a950b71ce\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") "
Sep 18 18:42:35.088878 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.088687    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wg7qd\" (UniqueName: \"kubernetes.io/projected/0e75115a-7692-43e9-90d3-438a950b71ce-kube-api-access-wg7qd\") pod \"0e75115a-7692-43e9-90d3-438a950b71ce\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") "
Sep 18 18:42:35.088878 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.088712    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e75115a-7692-43e9-90d3-438a950b71ce-console-oauth-config\") pod \"0e75115a-7692-43e9-90d3-438a950b71ce\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") "
Sep 18 18:42:35.088878 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.088748    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-console-config\") pod \"0e75115a-7692-43e9-90d3-438a950b71ce\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") "
Sep 18 18:42:35.088878 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.088774    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e75115a-7692-43e9-90d3-438a950b71ce-console-serving-cert\") pod \"0e75115a-7692-43e9-90d3-438a950b71ce\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") "
Sep 18 18:42:35.088878 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.088833    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-trusted-ca-bundle\") pod \"0e75115a-7692-43e9-90d3-438a950b71ce\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") "
Sep 18 18:42:35.089190 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.088893    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-oauth-serving-cert\") pod \"0e75115a-7692-43e9-90d3-438a950b71ce\" (UID: \"0e75115a-7692-43e9-90d3-438a950b71ce\") "
Sep 18 18:42:35.089248 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.089228    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-service-ca" (OuterVolumeSpecName: "service-ca") pod "0e75115a-7692-43e9-90d3-438a950b71ce" (UID: "0e75115a-7692-43e9-90d3-438a950b71ce"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:42:35.089299 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.089254    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-console-config" (OuterVolumeSpecName: "console-config") pod "0e75115a-7692-43e9-90d3-438a950b71ce" (UID: "0e75115a-7692-43e9-90d3-438a950b71ce"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:42:35.089378 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.089348    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0e75115a-7692-43e9-90d3-438a950b71ce" (UID: "0e75115a-7692-43e9-90d3-438a950b71ce"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:42:35.089454 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.089429    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0e75115a-7692-43e9-90d3-438a950b71ce" (UID: "0e75115a-7692-43e9-90d3-438a950b71ce"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:42:35.091177 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.091155    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e75115a-7692-43e9-90d3-438a950b71ce-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0e75115a-7692-43e9-90d3-438a950b71ce" (UID: "0e75115a-7692-43e9-90d3-438a950b71ce"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:42:35.091393 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.091375    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e75115a-7692-43e9-90d3-438a950b71ce-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0e75115a-7692-43e9-90d3-438a950b71ce" (UID: "0e75115a-7692-43e9-90d3-438a950b71ce"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:42:35.091457 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.091418    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e75115a-7692-43e9-90d3-438a950b71ce-kube-api-access-wg7qd" (OuterVolumeSpecName: "kube-api-access-wg7qd") pod "0e75115a-7692-43e9-90d3-438a950b71ce" (UID: "0e75115a-7692-43e9-90d3-438a950b71ce"). InnerVolumeSpecName "kube-api-access-wg7qd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:42:35.190056 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.190029    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-console-config\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:42:35.190056 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.190053    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e75115a-7692-43e9-90d3-438a950b71ce-console-serving-cert\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:42:35.190206 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.190066    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-trusted-ca-bundle\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:42:35.190206 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.190081    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-oauth-serving-cert\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:42:35.190206 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.190092    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e75115a-7692-43e9-90d3-438a950b71ce-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:42:35.190206 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.190105    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wg7qd\" (UniqueName: \"kubernetes.io/projected/0e75115a-7692-43e9-90d3-438a950b71ce-kube-api-access-wg7qd\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:42:35.190206 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.190117    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e75115a-7692-43e9-90d3-438a950b71ce-console-oauth-config\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:42:35.766111 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.766073    2558 generic.go:356] "Generic (PLEG): container finished" podID="0e75115a-7692-43e9-90d3-438a950b71ce" containerID="0bc44bff02ccc4a4a6cf68ed49fbae6e5ab7abc7c51f6457a0addf20dac3b82f" exitCode=0
Sep 18 18:42:35.766505 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.766121    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c9cccb589-l6j8t"
Sep 18 18:42:35.766505 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.766160    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c9cccb589-l6j8t" event={"ID":"0e75115a-7692-43e9-90d3-438a950b71ce","Type":"ContainerDied","Data":"0bc44bff02ccc4a4a6cf68ed49fbae6e5ab7abc7c51f6457a0addf20dac3b82f"}
Sep 18 18:42:35.766505 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.766211    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c9cccb589-l6j8t" event={"ID":"0e75115a-7692-43e9-90d3-438a950b71ce","Type":"ContainerDied","Data":"c8d5de393c81498b381df9ccd9c9be84a21e5793ccabb4d2a6c005bbec6b86e2"}
Sep 18 18:42:35.766505 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.766232    2558 scope.go:117] "RemoveContainer" containerID="0bc44bff02ccc4a4a6cf68ed49fbae6e5ab7abc7c51f6457a0addf20dac3b82f"
Sep 18 18:42:35.774922 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.774909    2558 scope.go:117] "RemoveContainer" containerID="0bc44bff02ccc4a4a6cf68ed49fbae6e5ab7abc7c51f6457a0addf20dac3b82f"
Sep 18 18:42:35.775155 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:42:35.775137    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bc44bff02ccc4a4a6cf68ed49fbae6e5ab7abc7c51f6457a0addf20dac3b82f\": container with ID starting with 0bc44bff02ccc4a4a6cf68ed49fbae6e5ab7abc7c51f6457a0addf20dac3b82f not found: ID does not exist" containerID="0bc44bff02ccc4a4a6cf68ed49fbae6e5ab7abc7c51f6457a0addf20dac3b82f"
Sep 18 18:42:35.775202 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.775162    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bc44bff02ccc4a4a6cf68ed49fbae6e5ab7abc7c51f6457a0addf20dac3b82f"} err="failed to get container status \"0bc44bff02ccc4a4a6cf68ed49fbae6e5ab7abc7c51f6457a0addf20dac3b82f\": rpc error: code = NotFound desc = could not find container \"0bc44bff02ccc4a4a6cf68ed49fbae6e5ab7abc7c51f6457a0addf20dac3b82f\": container with ID starting with 0bc44bff02ccc4a4a6cf68ed49fbae6e5ab7abc7c51f6457a0addf20dac3b82f not found: ID does not exist"
Sep 18 18:42:35.786404 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.786380    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c9cccb589-l6j8t"]
Sep 18 18:42:35.790020 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.789993    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c9cccb589-l6j8t"]
Sep 18 18:42:35.843167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:35.843143    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e75115a-7692-43e9-90d3-438a950b71ce" path="/var/lib/kubelet/pods/0e75115a-7692-43e9-90d3-438a950b71ce/volumes"
Sep 18 18:42:46.125503 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:46.125467    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:42:46.182002 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:46.181974    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:42:54.957911 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:54.957879    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:42:54.972884 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:54.972862    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:42:55.860788 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:42:55.860763    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:43:03.885953 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:03.885904    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:43:03.940200 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:03.940164    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:43:08.860105 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:08.860071    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-868bbdd6c7-jmqkc"]
Sep 18 18:43:08.860557 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:08.860445    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e75115a-7692-43e9-90d3-438a950b71ce" containerName="console"
Sep 18 18:43:08.860557 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:08.860459    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e75115a-7692-43e9-90d3-438a950b71ce" containerName="console"
Sep 18 18:43:08.860557 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:08.860520    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e75115a-7692-43e9-90d3-438a950b71ce" containerName="console"
Sep 18 18:43:08.863489 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:08.863474    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:08.872178 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:08.872158    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-868bbdd6c7-jmqkc"]
Sep 18 18:43:08.902921 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:08.902888    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:43:08.958588 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:08.958560    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:43:08.966545 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:08.966524    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a07e9367-6455-45c3-ba80-ab4765ec5cf8-console-serving-cert\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:08.966606 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:08.966550    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-service-ca\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:08.966606 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:08.966583    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-trusted-ca-bundle\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:08.966689 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:08.966654    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-oauth-serving-cert\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:08.966730 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:08.966692    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjrxt\" (UniqueName: \"kubernetes.io/projected/a07e9367-6455-45c3-ba80-ab4765ec5cf8-kube-api-access-sjrxt\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:08.966730 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:08.966713    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a07e9367-6455-45c3-ba80-ab4765ec5cf8-console-oauth-config\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:08.966730 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:08.966726    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-console-config\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:09.067814 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.067792    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-trusted-ca-bundle\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:09.067910 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.067841    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-oauth-serving-cert\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:09.067910 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.067867    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjrxt\" (UniqueName: \"kubernetes.io/projected/a07e9367-6455-45c3-ba80-ab4765ec5cf8-kube-api-access-sjrxt\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:09.068038 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.068023    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a07e9367-6455-45c3-ba80-ab4765ec5cf8-console-oauth-config\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:09.068083 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.068048    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-console-config\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:09.068194 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.068170    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a07e9367-6455-45c3-ba80-ab4765ec5cf8-console-serving-cert\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:09.068235 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.068216    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-service-ca\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:09.068598 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.068580    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-oauth-serving-cert\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:09.068677 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.068656    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-console-config\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:09.068761 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.068740    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-trusted-ca-bundle\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:09.068880 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.068863    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-service-ca\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:09.070560 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.070539    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a07e9367-6455-45c3-ba80-ab4765ec5cf8-console-serving-cert\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:09.070635 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.070547    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a07e9367-6455-45c3-ba80-ab4765ec5cf8-console-oauth-config\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:09.075339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.075318    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjrxt\" (UniqueName: \"kubernetes.io/projected/a07e9367-6455-45c3-ba80-ab4765ec5cf8-kube-api-access-sjrxt\") pod \"console-868bbdd6c7-jmqkc\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") " pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:09.173632 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.173576    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:09.444155 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.444131    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-868bbdd6c7-jmqkc"]
Sep 18 18:43:09.446689 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:43:09.446662    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda07e9367_6455_45c3_ba80_ab4765ec5cf8.slice/crio-337373ad43e66f21e9987e1810ab3cf4b5bc9c1c58676c9a5c98041ee3dd8829 WatchSource:0}: Error finding container 337373ad43e66f21e9987e1810ab3cf4b5bc9c1c58676c9a5c98041ee3dd8829: Status 404 returned error can't find the container with id 337373ad43e66f21e9987e1810ab3cf4b5bc9c1c58676c9a5c98041ee3dd8829
Sep 18 18:43:09.905921 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.905878    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-868bbdd6c7-jmqkc" event={"ID":"a07e9367-6455-45c3-ba80-ab4765ec5cf8","Type":"ContainerStarted","Data":"97026b9ff67026f79174079fedcbd07275af23ff919fceec75c3f2898b38e196"}
Sep 18 18:43:09.905921 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.905933    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-868bbdd6c7-jmqkc" event={"ID":"a07e9367-6455-45c3-ba80-ab4765ec5cf8","Type":"ContainerStarted","Data":"337373ad43e66f21e9987e1810ab3cf4b5bc9c1c58676c9a5c98041ee3dd8829"}
Sep 18 18:43:09.923910 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:09.923863    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-868bbdd6c7-jmqkc" podStartSLOduration=1.9238483720000001 podStartE2EDuration="1.923848372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:43:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:43:09.922797883 +0000 UTC m=+226.121282125" watchObservedRunningTime="2026-09-18 18:43:09.923848372 +0000 UTC m=+226.122332576"
Sep 18 18:43:15.879844 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:15.879802    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:43:15.932356 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:15.932327    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:43:19.173792 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:19.173747    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:19.174225 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:19.173813    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:19.178435 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:19.178417    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:19.948266 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:19.948233    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:43:19.992643 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:19.992606    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-566d77f5c7-569tf"]
Sep 18 18:43:20.104186 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:20.104154    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:43:20.204122 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:20.204047    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:43:30.708717 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:30.708678    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:43:30.809745 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:30.809709    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:43:31.132612 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:31.132577    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:43:31.231596 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:31.231563    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:43:45.011730 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.011637    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-566d77f5c7-569tf" podUID="4ef4bbee-1cc0-4f13-871b-64edcc003c16" containerName="console" containerID="cri-o://681dd0dc44e43bdff5cccb50f7b966589e738bf48a405b684e2afb7228b45a7e" gracePeriod=15
Sep 18 18:43:45.257728 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.257703    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:43:45.383959 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.383893    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4ef4bbee-1cc0-4f13-871b-64edcc003c16-console-serving-cert\") pod \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") "
Sep 18 18:43:45.383959 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.383934    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-console-config\") pod \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") "
Sep 18 18:43:45.384146 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.383961    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-service-ca\") pod \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") "
Sep 18 18:43:45.384146 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.383993    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-oauth-serving-cert\") pod \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") "
Sep 18 18:43:45.384146 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.384032    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-trusted-ca-bundle\") pod \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") "
Sep 18 18:43:45.384146 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.384114    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fpjx\" (UniqueName: \"kubernetes.io/projected/4ef4bbee-1cc0-4f13-871b-64edcc003c16-kube-api-access-4fpjx\") pod \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") "
Sep 18 18:43:45.384146 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.384141    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4ef4bbee-1cc0-4f13-871b-64edcc003c16-console-oauth-config\") pod \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\" (UID: \"4ef4bbee-1cc0-4f13-871b-64edcc003c16\") "
Sep 18 18:43:45.384474 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.384434    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-service-ca" (OuterVolumeSpecName: "service-ca") pod "4ef4bbee-1cc0-4f13-871b-64edcc003c16" (UID: "4ef4bbee-1cc0-4f13-871b-64edcc003c16"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:43:45.384609 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.384476    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-console-config" (OuterVolumeSpecName: "console-config") pod "4ef4bbee-1cc0-4f13-871b-64edcc003c16" (UID: "4ef4bbee-1cc0-4f13-871b-64edcc003c16"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:43:45.384609 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.384515    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:43:45.384609 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.384559    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4ef4bbee-1cc0-4f13-871b-64edcc003c16" (UID: "4ef4bbee-1cc0-4f13-871b-64edcc003c16"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:43:45.384787 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.384610    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4ef4bbee-1cc0-4f13-871b-64edcc003c16" (UID: "4ef4bbee-1cc0-4f13-871b-64edcc003c16"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:43:45.386192 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.386170    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef4bbee-1cc0-4f13-871b-64edcc003c16-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4ef4bbee-1cc0-4f13-871b-64edcc003c16" (UID: "4ef4bbee-1cc0-4f13-871b-64edcc003c16"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:43:45.386274 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.386249    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef4bbee-1cc0-4f13-871b-64edcc003c16-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4ef4bbee-1cc0-4f13-871b-64edcc003c16" (UID: "4ef4bbee-1cc0-4f13-871b-64edcc003c16"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:43:45.386340 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.386316    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ef4bbee-1cc0-4f13-871b-64edcc003c16-kube-api-access-4fpjx" (OuterVolumeSpecName: "kube-api-access-4fpjx") pod "4ef4bbee-1cc0-4f13-871b-64edcc003c16" (UID: "4ef4bbee-1cc0-4f13-871b-64edcc003c16"). InnerVolumeSpecName "kube-api-access-4fpjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:43:45.485406 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.485368    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-console-config\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:43:45.485406 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.485403    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-oauth-serving-cert\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:43:45.485559 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.485417    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ef4bbee-1cc0-4f13-871b-64edcc003c16-trusted-ca-bundle\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:43:45.485559 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.485430    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fpjx\" (UniqueName: \"kubernetes.io/projected/4ef4bbee-1cc0-4f13-871b-64edcc003c16-kube-api-access-4fpjx\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:43:45.485559 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.485443    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4ef4bbee-1cc0-4f13-871b-64edcc003c16-console-oauth-config\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:43:45.485559 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:45.485454    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4ef4bbee-1cc0-4f13-871b-64edcc003c16-console-serving-cert\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:43:46.050814 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:46.050782    2558 generic.go:356] "Generic (PLEG): container finished" podID="4ef4bbee-1cc0-4f13-871b-64edcc003c16" containerID="681dd0dc44e43bdff5cccb50f7b966589e738bf48a405b684e2afb7228b45a7e" exitCode=0
Sep 18 18:43:46.050814 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:46.050835    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-566d77f5c7-569tf"
Sep 18 18:43:46.051280 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:46.050848    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-566d77f5c7-569tf" event={"ID":"4ef4bbee-1cc0-4f13-871b-64edcc003c16","Type":"ContainerDied","Data":"681dd0dc44e43bdff5cccb50f7b966589e738bf48a405b684e2afb7228b45a7e"}
Sep 18 18:43:46.051280 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:46.050899    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-566d77f5c7-569tf" event={"ID":"4ef4bbee-1cc0-4f13-871b-64edcc003c16","Type":"ContainerDied","Data":"00b8243e0a4a4acba070530aa967b2c23c5155c54314ae801ade6e9672c940aa"}
Sep 18 18:43:46.051280 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:46.050919    2558 scope.go:117] "RemoveContainer" containerID="681dd0dc44e43bdff5cccb50f7b966589e738bf48a405b684e2afb7228b45a7e"
Sep 18 18:43:46.058991 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:46.058969    2558 scope.go:117] "RemoveContainer" containerID="681dd0dc44e43bdff5cccb50f7b966589e738bf48a405b684e2afb7228b45a7e"
Sep 18 18:43:46.059248 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:43:46.059231    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"681dd0dc44e43bdff5cccb50f7b966589e738bf48a405b684e2afb7228b45a7e\": container with ID starting with 681dd0dc44e43bdff5cccb50f7b966589e738bf48a405b684e2afb7228b45a7e not found: ID does not exist" containerID="681dd0dc44e43bdff5cccb50f7b966589e738bf48a405b684e2afb7228b45a7e"
Sep 18 18:43:46.059301 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:46.059259    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"681dd0dc44e43bdff5cccb50f7b966589e738bf48a405b684e2afb7228b45a7e"} err="failed to get container status \"681dd0dc44e43bdff5cccb50f7b966589e738bf48a405b684e2afb7228b45a7e\": rpc error: code = NotFound desc = could not find container \"681dd0dc44e43bdff5cccb50f7b966589e738bf48a405b684e2afb7228b45a7e\": container with ID starting with 681dd0dc44e43bdff5cccb50f7b966589e738bf48a405b684e2afb7228b45a7e not found: ID does not exist"
Sep 18 18:43:46.067593 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:46.067563    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-566d77f5c7-569tf"]
Sep 18 18:43:46.072765 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:46.072746    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-566d77f5c7-569tf"]
Sep 18 18:43:47.843384 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:43:47.843353    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ef4bbee-1cc0-4f13-871b-64edcc003c16" path="/var/lib/kubelet/pods/4ef4bbee-1cc0-4f13-871b-64edcc003c16/volumes"
Sep 18 18:44:10.822039 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:10.821996    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:44:12.017903 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:12.017870    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:44:25.711535 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:25.711507    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bwsg_636f0979-fe2e-49fd-86d6-31bd7961e736/ovn-acl-logging/0.log"
Sep 18 18:44:25.715243 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:25.715216    2558 kubelet.go:1648] "Image garbage collection succeeded"
Sep 18 18:44:25.739127 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:25.738959    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bwsg_636f0979-fe2e-49fd-86d6-31bd7961e736/ovn-acl-logging/0.log"
Sep 18 18:44:37.783969 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:37.783932    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:44:39.194309 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:39.194274    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:44:41.402895 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:41.402864    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:44:41.503434 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:41.503403    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:44:43.326242 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:43.326210    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:44:43.409394 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:43.409365    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dtnzp"]
Sep 18 18:44:43.409687 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:43.409676    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ef4bbee-1cc0-4f13-871b-64edcc003c16" containerName="console"
Sep 18 18:44:43.409731 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:43.409689    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ef4bbee-1cc0-4f13-871b-64edcc003c16" containerName="console"
Sep 18 18:44:43.409765 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:43.409738    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ef4bbee-1cc0-4f13-871b-64edcc003c16" containerName="console"
Sep 18 18:44:43.414334 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:43.414302    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-dtnzp"
Sep 18 18:44:43.418150 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:43.418122    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 18 18:44:43.418277 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:43.418237    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 18 18:44:43.427005 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:43.426984    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dtnzp"]
Sep 18 18:44:43.429979 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:43.429946    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:44:43.539598 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:43.539558    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:44:43.541542 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:43.541520    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf729\" (UniqueName: \"kubernetes.io/projected/99b27708-dd5f-4348-85d0-85d9b4b0780e-kube-api-access-sf729\") pod \"cert-manager-webhook-6486ffd997-dtnzp\" (UID: \"99b27708-dd5f-4348-85d0-85d9b4b0780e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dtnzp"
Sep 18 18:44:43.541622 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:43.541552    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99b27708-dd5f-4348-85d0-85d9b4b0780e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dtnzp\" (UID: \"99b27708-dd5f-4348-85d0-85d9b4b0780e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dtnzp"
Sep 18 18:44:43.642330 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:43.642261    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf729\" (UniqueName: \"kubernetes.io/projected/99b27708-dd5f-4348-85d0-85d9b4b0780e-kube-api-access-sf729\") pod \"cert-manager-webhook-6486ffd997-dtnzp\" (UID: \"99b27708-dd5f-4348-85d0-85d9b4b0780e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dtnzp"
Sep 18 18:44:43.642330 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:43.642292    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99b27708-dd5f-4348-85d0-85d9b4b0780e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dtnzp\" (UID: \"99b27708-dd5f-4348-85d0-85d9b4b0780e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dtnzp"
Sep 18 18:44:43.661035 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:43.661011    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99b27708-dd5f-4348-85d0-85d9b4b0780e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dtnzp\" (UID: \"99b27708-dd5f-4348-85d0-85d9b4b0780e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dtnzp"
Sep 18 18:44:43.661162 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:43.661127    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf729\" (UniqueName: \"kubernetes.io/projected/99b27708-dd5f-4348-85d0-85d9b4b0780e-kube-api-access-sf729\") pod \"cert-manager-webhook-6486ffd997-dtnzp\" (UID: \"99b27708-dd5f-4348-85d0-85d9b4b0780e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dtnzp"
Sep 18 18:44:43.731235 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:43.731203    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-dtnzp"
Sep 18 18:44:44.004121 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:44.004028    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dtnzp"]
Sep 18 18:44:44.006846 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:44:44.006812    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99b27708_dd5f_4348_85d0_85d9b4b0780e.slice/crio-02ee4fa3ec31a1c42ab65dd3e1d125c9ebb6633d665fb722969dccc1e05d63ce WatchSource:0}: Error finding container 02ee4fa3ec31a1c42ab65dd3e1d125c9ebb6633d665fb722969dccc1e05d63ce: Status 404 returned error can't find the container with id 02ee4fa3ec31a1c42ab65dd3e1d125c9ebb6633d665fb722969dccc1e05d63ce
Sep 18 18:44:44.008746 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:44.008731    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:44:44.282352 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:44.282286    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-dtnzp" event={"ID":"99b27708-dd5f-4348-85d0-85d9b4b0780e","Type":"ContainerStarted","Data":"02ee4fa3ec31a1c42ab65dd3e1d125c9ebb6633d665fb722969dccc1e05d63ce"}
Sep 18 18:44:45.471963 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:45.471927    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:44:45.887796 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:45.887756    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:44:47.298527 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:47.298490    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-dtnzp" event={"ID":"99b27708-dd5f-4348-85d0-85d9b4b0780e","Type":"ContainerStarted","Data":"9a5cf0473c183a674125262a142325626269f4f1f9d01d5e07259f2425f40fe6"}
Sep 18 18:44:47.298902 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:47.298610    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-dtnzp"
Sep 18 18:44:47.316203 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:47.316160    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-dtnzp" podStartSLOduration=1.193311911 podStartE2EDuration="4.316146142s" totalImagesPullingTime="3.122834231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:44:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:47.315240087 +0000 UTC m=+323.513724293" watchObservedRunningTime="2026-09-18 18:44:47.316146142 +0000 UTC m=+323.514630346"
Sep 18 18:44:48.398985 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:48.398948    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:44:48.497346 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:48.497312    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:44:53.278028 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:53.277993    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:44:53.308994 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:53.308969    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-dtnzp"
Sep 18 18:44:53.808856 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:53.808810    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-s2997"]
Sep 18 18:44:53.812117 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:53.812103    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-s2997"
Sep 18 18:44:53.814465 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:53.814444    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-wjqwx\""
Sep 18 18:44:53.820118 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:53.820094    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-s2997"]
Sep 18 18:44:53.926767 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:53.926735    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97ee5609-9eb4-41cb-96de-15b808461091-bound-sa-token\") pod \"cert-manager-55b4d954d5-s2997\" (UID: \"97ee5609-9eb4-41cb-96de-15b808461091\") " pod="cert-manager/cert-manager-55b4d954d5-s2997"
Sep 18 18:44:53.926767 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:53.926768    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp5v4\" (UniqueName: \"kubernetes.io/projected/97ee5609-9eb4-41cb-96de-15b808461091-kube-api-access-fp5v4\") pod \"cert-manager-55b4d954d5-s2997\" (UID: \"97ee5609-9eb4-41cb-96de-15b808461091\") " pod="cert-manager/cert-manager-55b4d954d5-s2997"
Sep 18 18:44:54.028156 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:54.028109    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97ee5609-9eb4-41cb-96de-15b808461091-bound-sa-token\") pod \"cert-manager-55b4d954d5-s2997\" (UID: \"97ee5609-9eb4-41cb-96de-15b808461091\") " pod="cert-manager/cert-manager-55b4d954d5-s2997"
Sep 18 18:44:54.028156 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:54.028162    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp5v4\" (UniqueName: \"kubernetes.io/projected/97ee5609-9eb4-41cb-96de-15b808461091-kube-api-access-fp5v4\") pod \"cert-manager-55b4d954d5-s2997\" (UID: \"97ee5609-9eb4-41cb-96de-15b808461091\") " pod="cert-manager/cert-manager-55b4d954d5-s2997"
Sep 18 18:44:54.036185 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:54.036162    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97ee5609-9eb4-41cb-96de-15b808461091-bound-sa-token\") pod \"cert-manager-55b4d954d5-s2997\" (UID: \"97ee5609-9eb4-41cb-96de-15b808461091\") " pod="cert-manager/cert-manager-55b4d954d5-s2997"
Sep 18 18:44:54.036387 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:54.036368    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp5v4\" (UniqueName: \"kubernetes.io/projected/97ee5609-9eb4-41cb-96de-15b808461091-kube-api-access-fp5v4\") pod \"cert-manager-55b4d954d5-s2997\" (UID: \"97ee5609-9eb4-41cb-96de-15b808461091\") " pod="cert-manager/cert-manager-55b4d954d5-s2997"
Sep 18 18:44:54.120927 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:54.120843    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-s2997"
Sep 18 18:44:54.391372 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:54.391344    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-s2997"]
Sep 18 18:44:54.393999 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:44:54.393974    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97ee5609_9eb4_41cb_96de_15b808461091.slice/crio-961b5d1ee389ab2bc50bfd378eac7fe83bd4aa67c65d659de087c49610eeb3a4 WatchSource:0}: Error finding container 961b5d1ee389ab2bc50bfd378eac7fe83bd4aa67c65d659de087c49610eeb3a4: Status 404 returned error can't find the container with id 961b5d1ee389ab2bc50bfd378eac7fe83bd4aa67c65d659de087c49610eeb3a4
Sep 18 18:44:55.338738 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:55.338705    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-s2997" event={"ID":"97ee5609-9eb4-41cb-96de-15b808461091","Type":"ContainerStarted","Data":"295a3bf16712ce1c3afb0824890a461326523005223bb9d61a12f01aab108c45"}
Sep 18 18:44:55.338738 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:55.338744    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-s2997" event={"ID":"97ee5609-9eb4-41cb-96de-15b808461091","Type":"ContainerStarted","Data":"961b5d1ee389ab2bc50bfd378eac7fe83bd4aa67c65d659de087c49610eeb3a4"}
Sep 18 18:44:55.354271 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:44:55.354226    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-s2997" podStartSLOduration=2.354213226 podStartE2EDuration="2.354213226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:44:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:55.35411793 +0000 UTC m=+331.552602134" watchObservedRunningTime="2026-09-18 18:44:55.354213226 +0000 UTC m=+331.552697431"
Sep 18 18:45:00.012532 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:00.012502    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:00.107205 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:00.107170    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:05.859934 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:05.859889    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:05.960245 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:05.960212    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:07.907235 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:07.907145    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:15.486131 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:15.486100    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:15.539130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:15.539101    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:21.310632 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:21.310596    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:21.415832 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:21.415781    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:21.890280 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:21.890245    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:21.989626 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:21.989590    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:22.509224 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:22.509184    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:22.689155 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:22.689126    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:22.785271 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:22.785180    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:22.906441 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:22.906404    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"]
Sep 18 18:45:22.911159 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:22.911136    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:22.914332 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:22.914310    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 18 18:45:22.914447 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:22.914428    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fnn27\""
Sep 18 18:45:22.914531 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:22.914508    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 18 18:45:22.914691 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:22.914675    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 18 18:45:22.914789 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:22.914772    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 18 18:45:22.936839 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:22.936802    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"]
Sep 18 18:45:22.977559 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:22.977531    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a391624-5f5e-4cff-9532-6a6703ac9d86-webhook-cert\") pod \"opendatahub-operator-controller-manager-55bcff78dd-v2xdw\" (UID: \"5a391624-5f5e-4cff-9532-6a6703ac9d86\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:22.977677 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:22.977566    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64dqn\" (UniqueName: \"kubernetes.io/projected/5a391624-5f5e-4cff-9532-6a6703ac9d86-kube-api-access-64dqn\") pod \"opendatahub-operator-controller-manager-55bcff78dd-v2xdw\" (UID: \"5a391624-5f5e-4cff-9532-6a6703ac9d86\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:22.977677 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:22.977600    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5a391624-5f5e-4cff-9532-6a6703ac9d86-manifests\") pod \"opendatahub-operator-controller-manager-55bcff78dd-v2xdw\" (UID: \"5a391624-5f5e-4cff-9532-6a6703ac9d86\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:22.977677 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:22.977624    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5a391624-5f5e-4cff-9532-6a6703ac9d86-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55bcff78dd-v2xdw\" (UID: \"5a391624-5f5e-4cff-9532-6a6703ac9d86\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:22.977677 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:22.977650    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a391624-5f5e-4cff-9532-6a6703ac9d86-tmp\") pod \"opendatahub-operator-controller-manager-55bcff78dd-v2xdw\" (UID: \"5a391624-5f5e-4cff-9532-6a6703ac9d86\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:23.053025 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:23.052915    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:23.079063 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:23.079043    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a391624-5f5e-4cff-9532-6a6703ac9d86-tmp\") pod \"opendatahub-operator-controller-manager-55bcff78dd-v2xdw\" (UID: \"5a391624-5f5e-4cff-9532-6a6703ac9d86\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:23.079165 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:23.079099    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a391624-5f5e-4cff-9532-6a6703ac9d86-webhook-cert\") pod \"opendatahub-operator-controller-manager-55bcff78dd-v2xdw\" (UID: \"5a391624-5f5e-4cff-9532-6a6703ac9d86\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:23.079165 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:23.079127    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64dqn\" (UniqueName: \"kubernetes.io/projected/5a391624-5f5e-4cff-9532-6a6703ac9d86-kube-api-access-64dqn\") pod \"opendatahub-operator-controller-manager-55bcff78dd-v2xdw\" (UID: \"5a391624-5f5e-4cff-9532-6a6703ac9d86\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:23.079165 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:23.079158    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5a391624-5f5e-4cff-9532-6a6703ac9d86-manifests\") pod \"opendatahub-operator-controller-manager-55bcff78dd-v2xdw\" (UID: \"5a391624-5f5e-4cff-9532-6a6703ac9d86\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:23.079307 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:23.079174    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5a391624-5f5e-4cff-9532-6a6703ac9d86-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55bcff78dd-v2xdw\" (UID: \"5a391624-5f5e-4cff-9532-6a6703ac9d86\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:23.079538 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:23.079521    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5a391624-5f5e-4cff-9532-6a6703ac9d86-manifests\") pod \"opendatahub-operator-controller-manager-55bcff78dd-v2xdw\" (UID: \"5a391624-5f5e-4cff-9532-6a6703ac9d86\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:23.081333 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:23.081306    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a391624-5f5e-4cff-9532-6a6703ac9d86-tmp\") pod \"opendatahub-operator-controller-manager-55bcff78dd-v2xdw\" (UID: \"5a391624-5f5e-4cff-9532-6a6703ac9d86\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:23.081429 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:23.081374    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a391624-5f5e-4cff-9532-6a6703ac9d86-webhook-cert\") pod \"opendatahub-operator-controller-manager-55bcff78dd-v2xdw\" (UID: \"5a391624-5f5e-4cff-9532-6a6703ac9d86\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:23.081475 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:23.081438    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5a391624-5f5e-4cff-9532-6a6703ac9d86-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55bcff78dd-v2xdw\" (UID: \"5a391624-5f5e-4cff-9532-6a6703ac9d86\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:23.093616 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:23.093595    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64dqn\" (UniqueName: \"kubernetes.io/projected/5a391624-5f5e-4cff-9532-6a6703ac9d86-kube-api-access-64dqn\") pod \"opendatahub-operator-controller-manager-55bcff78dd-v2xdw\" (UID: \"5a391624-5f5e-4cff-9532-6a6703ac9d86\") " pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:23.153472 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:23.153450    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:23.222519 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:23.222486    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:23.253119 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:23.253077    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:23.354974 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:23.354877    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:23.443454 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:23.443416    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:23.506570 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:23.506544    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"]
Sep 18 18:45:23.509106 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:45:23.509076    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a391624_5f5e_4cff_9532_6a6703ac9d86.slice/crio-ef4bbcc711e8ad1b54205f03cd89a01e7c7a34930cba918c5db916420bf107f0 WatchSource:0}: Error finding container ef4bbcc711e8ad1b54205f03cd89a01e7c7a34930cba918c5db916420bf107f0: Status 404 returned error can't find the container with id ef4bbcc711e8ad1b54205f03cd89a01e7c7a34930cba918c5db916420bf107f0
Sep 18 18:45:23.509604 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:23.509577    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:24.459713 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:24.459675    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw" event={"ID":"5a391624-5f5e-4cff-9532-6a6703ac9d86","Type":"ContainerStarted","Data":"ef4bbcc711e8ad1b54205f03cd89a01e7c7a34930cba918c5db916420bf107f0"}
Sep 18 18:45:25.773995 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.773959    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-569c544cc6-r5wff"]
Sep 18 18:45:25.776362 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.776338    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.800074 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.800046    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-569c544cc6-r5wff"]
Sep 18 18:45:25.804476 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.804433    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/52d908ad-92a4-4cdb-a279-c71290e038b3-service-ca\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.804592 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.804487    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/52d908ad-92a4-4cdb-a279-c71290e038b3-console-config\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.804592 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.804513    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52d908ad-92a4-4cdb-a279-c71290e038b3-trusted-ca-bundle\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.804592 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.804549    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqx7t\" (UniqueName: \"kubernetes.io/projected/52d908ad-92a4-4cdb-a279-c71290e038b3-kube-api-access-cqx7t\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.804592 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.804578    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/52d908ad-92a4-4cdb-a279-c71290e038b3-console-serving-cert\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.805158 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.804638    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/52d908ad-92a4-4cdb-a279-c71290e038b3-oauth-serving-cert\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.805158 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.804717    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/52d908ad-92a4-4cdb-a279-c71290e038b3-console-oauth-config\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.839721 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.839640    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:25.906208 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.906173    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/52d908ad-92a4-4cdb-a279-c71290e038b3-service-ca\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.906389 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.906225    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/52d908ad-92a4-4cdb-a279-c71290e038b3-console-config\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.906389 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.906337    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52d908ad-92a4-4cdb-a279-c71290e038b3-trusted-ca-bundle\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.906389 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.906364    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqx7t\" (UniqueName: \"kubernetes.io/projected/52d908ad-92a4-4cdb-a279-c71290e038b3-kube-api-access-cqx7t\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.906553 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.906405    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/52d908ad-92a4-4cdb-a279-c71290e038b3-console-serving-cert\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.906553 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.906427    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/52d908ad-92a4-4cdb-a279-c71290e038b3-oauth-serving-cert\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.906553 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.906457    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/52d908ad-92a4-4cdb-a279-c71290e038b3-console-oauth-config\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.907068 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.907032    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/52d908ad-92a4-4cdb-a279-c71290e038b3-service-ca\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.907188 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.907093    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/52d908ad-92a4-4cdb-a279-c71290e038b3-console-config\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.907254 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.907227    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52d908ad-92a4-4cdb-a279-c71290e038b3-trusted-ca-bundle\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.907437 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.907399    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/52d908ad-92a4-4cdb-a279-c71290e038b3-oauth-serving-cert\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.909179 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.909154    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/52d908ad-92a4-4cdb-a279-c71290e038b3-console-oauth-config\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.909521 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.909497    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/52d908ad-92a4-4cdb-a279-c71290e038b3-console-serving-cert\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.915628 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.915609    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqx7t\" (UniqueName: \"kubernetes.io/projected/52d908ad-92a4-4cdb-a279-c71290e038b3-kube-api-access-cqx7t\") pod \"console-569c544cc6-r5wff\" (UID: \"52d908ad-92a4-4cdb-a279-c71290e038b3\") " pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:25.938597 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:25.938566    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:26.088312 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:26.088235    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:26.231747 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:26.231710    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:26.344237 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:26.344150    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:26.769171 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:26.769123    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-569c544cc6-r5wff"]
Sep 18 18:45:26.771812 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:45:26.771789    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52d908ad_92a4_4cdb_a279_c71290e038b3.slice/crio-ab8d773def7530c5a166c9bf4e4a60734ef1c96d4169737a8212f60d63200f62 WatchSource:0}: Error finding container ab8d773def7530c5a166c9bf4e4a60734ef1c96d4169737a8212f60d63200f62: Status 404 returned error can't find the container with id ab8d773def7530c5a166c9bf4e4a60734ef1c96d4169737a8212f60d63200f62
Sep 18 18:45:27.476093 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:27.476057    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw" event={"ID":"5a391624-5f5e-4cff-9532-6a6703ac9d86","Type":"ContainerStarted","Data":"b9909d68c965459d777c82bc8e4506a47d2f1643c98a2647cf126b21d416659b"}
Sep 18 18:45:27.477658 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:27.477634    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-569c544cc6-r5wff" event={"ID":"52d908ad-92a4-4cdb-a279-c71290e038b3","Type":"ContainerStarted","Data":"a17c5af7da8f9537c29e18498962681e76aff24c98127a7f2d25074103a88203"}
Sep 18 18:45:27.477878 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:27.477679    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-569c544cc6-r5wff" event={"ID":"52d908ad-92a4-4cdb-a279-c71290e038b3","Type":"ContainerStarted","Data":"ab8d773def7530c5a166c9bf4e4a60734ef1c96d4169737a8212f60d63200f62"}
Sep 18 18:45:27.522897 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:27.522854    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-569c544cc6-r5wff" podStartSLOduration=2.522839331 podStartE2EDuration="2.522839331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:27.522698979 +0000 UTC m=+363.721183218" watchObservedRunningTime="2026-09-18 18:45:27.522839331 +0000 UTC m=+363.721323532"
Sep 18 18:45:28.483127 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:28.483092    2558 generic.go:356] "Generic (PLEG): container finished" podID="5a391624-5f5e-4cff-9532-6a6703ac9d86" containerID="b9909d68c965459d777c82bc8e4506a47d2f1643c98a2647cf126b21d416659b" exitCode=0
Sep 18 18:45:28.483499 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:28.483179    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw" event={"ID":"5a391624-5f5e-4cff-9532-6a6703ac9d86","Type":"ContainerDied","Data":"b9909d68c965459d777c82bc8e4506a47d2f1643c98a2647cf126b21d416659b"}
Sep 18 18:45:29.489342 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:29.489309    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw" event={"ID":"5a391624-5f5e-4cff-9532-6a6703ac9d86","Type":"ContainerStarted","Data":"37029f755cab1bd673aa9b4726dee92a46b73a3c41d5c02a9f1bde6b2abf81c4"}
Sep 18 18:45:29.489843 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:29.489422    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:29.512358 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:29.512314    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw" podStartSLOduration=3.063738394 podStartE2EDuration="7.512299534s" totalImagesPullingTime="3.427010548s" totalInitContainerRuntime="1.021550592s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:29.510400809 +0000 UTC m=+365.708885027" watchObservedRunningTime="2026-09-18 18:45:29.512299534 +0000 UTC m=+365.710783737"
Sep 18 18:45:36.088594 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:36.088564    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:36.088987 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:36.088678    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:36.092962 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:36.092944    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:36.522852 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:36.522814    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-569c544cc6-r5wff"
Sep 18 18:45:36.566739 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:36.566704    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-868bbdd6c7-jmqkc"]
Sep 18 18:45:36.680257 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:36.680227    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:36.778809 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:36.778729    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:36.839894 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:36.839865    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:40.496206 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.496176    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-55bcff78dd-v2xdw"
Sep 18 18:45:40.596151 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.596122    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"]
Sep 18 18:45:40.598443 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.598428    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"
Sep 18 18:45:40.601634 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.601610    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 18 18:45:40.601748 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.601666    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 18 18:45:40.601748 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.601607    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 18 18:45:40.601748 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.601723    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 18 18:45:40.601963 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.601612    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:45:40.602026 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.602010    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vdzdx\""
Sep 18 18:45:40.608654 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.608634    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"]
Sep 18 18:45:40.727195 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.727163    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2e96bcb4-8ab1-4777-93fc-03b873fe4442-manager-config\") pod \"lws-controller-manager-7cddfd44f8-mhnnj\" (UID: \"2e96bcb4-8ab1-4777-93fc-03b873fe4442\") " pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"
Sep 18 18:45:40.727355 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.727217    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e96bcb4-8ab1-4777-93fc-03b873fe4442-metrics-cert\") pod \"lws-controller-manager-7cddfd44f8-mhnnj\" (UID: \"2e96bcb4-8ab1-4777-93fc-03b873fe4442\") " pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"
Sep 18 18:45:40.727355 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.727234    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2fdb\" (UniqueName: \"kubernetes.io/projected/2e96bcb4-8ab1-4777-93fc-03b873fe4442-kube-api-access-d2fdb\") pod \"lws-controller-manager-7cddfd44f8-mhnnj\" (UID: \"2e96bcb4-8ab1-4777-93fc-03b873fe4442\") " pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"
Sep 18 18:45:40.727355 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.727345    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e96bcb4-8ab1-4777-93fc-03b873fe4442-cert\") pod \"lws-controller-manager-7cddfd44f8-mhnnj\" (UID: \"2e96bcb4-8ab1-4777-93fc-03b873fe4442\") " pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"
Sep 18 18:45:40.828483 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.828420    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e96bcb4-8ab1-4777-93fc-03b873fe4442-cert\") pod \"lws-controller-manager-7cddfd44f8-mhnnj\" (UID: \"2e96bcb4-8ab1-4777-93fc-03b873fe4442\") " pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"
Sep 18 18:45:40.828483 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.828453    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2e96bcb4-8ab1-4777-93fc-03b873fe4442-manager-config\") pod \"lws-controller-manager-7cddfd44f8-mhnnj\" (UID: \"2e96bcb4-8ab1-4777-93fc-03b873fe4442\") " pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"
Sep 18 18:45:40.828682 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.828512    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e96bcb4-8ab1-4777-93fc-03b873fe4442-metrics-cert\") pod \"lws-controller-manager-7cddfd44f8-mhnnj\" (UID: \"2e96bcb4-8ab1-4777-93fc-03b873fe4442\") " pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"
Sep 18 18:45:40.828682 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.828542    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2fdb\" (UniqueName: \"kubernetes.io/projected/2e96bcb4-8ab1-4777-93fc-03b873fe4442-kube-api-access-d2fdb\") pod \"lws-controller-manager-7cddfd44f8-mhnnj\" (UID: \"2e96bcb4-8ab1-4777-93fc-03b873fe4442\") " pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"
Sep 18 18:45:40.829204 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.829183    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2e96bcb4-8ab1-4777-93fc-03b873fe4442-manager-config\") pod \"lws-controller-manager-7cddfd44f8-mhnnj\" (UID: \"2e96bcb4-8ab1-4777-93fc-03b873fe4442\") " pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"
Sep 18 18:45:40.830915 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.830888    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e96bcb4-8ab1-4777-93fc-03b873fe4442-cert\") pod \"lws-controller-manager-7cddfd44f8-mhnnj\" (UID: \"2e96bcb4-8ab1-4777-93fc-03b873fe4442\") " pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"
Sep 18 18:45:40.831072 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.831053    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e96bcb4-8ab1-4777-93fc-03b873fe4442-metrics-cert\") pod \"lws-controller-manager-7cddfd44f8-mhnnj\" (UID: \"2e96bcb4-8ab1-4777-93fc-03b873fe4442\") " pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"
Sep 18 18:45:40.836837 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.836796    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2fdb\" (UniqueName: \"kubernetes.io/projected/2e96bcb4-8ab1-4777-93fc-03b873fe4442-kube-api-access-d2fdb\") pod \"lws-controller-manager-7cddfd44f8-mhnnj\" (UID: \"2e96bcb4-8ab1-4777-93fc-03b873fe4442\") " pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"
Sep 18 18:45:40.908575 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:40.908549    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"
Sep 18 18:45:41.178657 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:41.178587    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"]
Sep 18 18:45:41.181669 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:45:41.181639    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e96bcb4_8ab1_4777_93fc_03b873fe4442.slice/crio-559bfeb77ab83372a751694b3563fd7d34b05774cb3f39dd8787f5bb18c5e54c WatchSource:0}: Error finding container 559bfeb77ab83372a751694b3563fd7d34b05774cb3f39dd8787f5bb18c5e54c: Status 404 returned error can't find the container with id 559bfeb77ab83372a751694b3563fd7d34b05774cb3f39dd8787f5bb18c5e54c
Sep 18 18:45:41.543163 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:41.543130    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj" event={"ID":"2e96bcb4-8ab1-4777-93fc-03b873fe4442","Type":"ContainerStarted","Data":"559bfeb77ab83372a751694b3563fd7d34b05774cb3f39dd8787f5bb18c5e54c"}
Sep 18 18:45:42.099192 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.099159    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:42.242221 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.242137    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:42.570515 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.570469    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:42.616368 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.616328    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-c66559bbb-jdc69"]
Sep 18 18:45:42.620725 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.620702    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c66559bbb-jdc69"
Sep 18 18:45:42.623691 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.623657    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 18 18:45:42.623813 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.623699    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 18 18:45:42.624882 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.624860    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mbwgc\""
Sep 18 18:45:42.624986 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.624879    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 18 18:45:42.624986 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.624902    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 18 18:45:42.633269 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.633249    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c66559bbb-jdc69"]
Sep 18 18:45:42.746833 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.746792    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvqvb\" (UniqueName: \"kubernetes.io/projected/7ef4bddc-01ec-4ee6-9a48-8f2e81586749-kube-api-access-vvqvb\") pod \"kube-auth-proxy-c66559bbb-jdc69\" (UID: \"7ef4bddc-01ec-4ee6-9a48-8f2e81586749\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-jdc69"
Sep 18 18:45:42.747011 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.746863    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ef4bddc-01ec-4ee6-9a48-8f2e81586749-tmp\") pod \"kube-auth-proxy-c66559bbb-jdc69\" (UID: \"7ef4bddc-01ec-4ee6-9a48-8f2e81586749\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-jdc69"
Sep 18 18:45:42.747011 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.746943    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ef4bddc-01ec-4ee6-9a48-8f2e81586749-tls-certs\") pod \"kube-auth-proxy-c66559bbb-jdc69\" (UID: \"7ef4bddc-01ec-4ee6-9a48-8f2e81586749\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-jdc69"
Sep 18 18:45:42.847966 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.847881    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ef4bddc-01ec-4ee6-9a48-8f2e81586749-tls-certs\") pod \"kube-auth-proxy-c66559bbb-jdc69\" (UID: \"7ef4bddc-01ec-4ee6-9a48-8f2e81586749\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-jdc69"
Sep 18 18:45:42.847966 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.847964    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvqvb\" (UniqueName: \"kubernetes.io/projected/7ef4bddc-01ec-4ee6-9a48-8f2e81586749-kube-api-access-vvqvb\") pod \"kube-auth-proxy-c66559bbb-jdc69\" (UID: \"7ef4bddc-01ec-4ee6-9a48-8f2e81586749\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-jdc69"
Sep 18 18:45:42.848178 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.848022    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ef4bddc-01ec-4ee6-9a48-8f2e81586749-tmp\") pod \"kube-auth-proxy-c66559bbb-jdc69\" (UID: \"7ef4bddc-01ec-4ee6-9a48-8f2e81586749\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-jdc69"
Sep 18 18:45:42.848178 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:45:42.848032    2558 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 18 18:45:42.848178 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:45:42.848100    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ef4bddc-01ec-4ee6-9a48-8f2e81586749-tls-certs podName:7ef4bddc-01ec-4ee6-9a48-8f2e81586749 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:43.348080538 +0000 UTC m=+379.546564722 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7ef4bddc-01ec-4ee6-9a48-8f2e81586749-tls-certs") pod "kube-auth-proxy-c66559bbb-jdc69" (UID: "7ef4bddc-01ec-4ee6-9a48-8f2e81586749") : secret "kube-auth-proxy-tls" not found
Sep 18 18:45:42.850443 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.850417    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ef4bddc-01ec-4ee6-9a48-8f2e81586749-tmp\") pod \"kube-auth-proxy-c66559bbb-jdc69\" (UID: \"7ef4bddc-01ec-4ee6-9a48-8f2e81586749\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-jdc69"
Sep 18 18:45:42.859599 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:42.859574    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvqvb\" (UniqueName: \"kubernetes.io/projected/7ef4bddc-01ec-4ee6-9a48-8f2e81586749-kube-api-access-vvqvb\") pod \"kube-auth-proxy-c66559bbb-jdc69\" (UID: \"7ef4bddc-01ec-4ee6-9a48-8f2e81586749\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-jdc69"
Sep 18 18:45:43.049199 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:43.049160    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:43.352983 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:43.352949    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ef4bddc-01ec-4ee6-9a48-8f2e81586749-tls-certs\") pod \"kube-auth-proxy-c66559bbb-jdc69\" (UID: \"7ef4bddc-01ec-4ee6-9a48-8f2e81586749\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-jdc69"
Sep 18 18:45:43.355651 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:43.355631    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ef4bddc-01ec-4ee6-9a48-8f2e81586749-tls-certs\") pod \"kube-auth-proxy-c66559bbb-jdc69\" (UID: \"7ef4bddc-01ec-4ee6-9a48-8f2e81586749\") " pod="openshift-ingress/kube-auth-proxy-c66559bbb-jdc69"
Sep 18 18:45:43.533462 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:43.533407    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c66559bbb-jdc69"
Sep 18 18:45:43.806812 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:43.806745    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c66559bbb-jdc69"]
Sep 18 18:45:43.809422 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:45:43.809395    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ef4bddc_01ec_4ee6_9a48_8f2e81586749.slice/crio-e7fa79f8c5f3306feedd76a1bc5448c0dd7465d551d14c74961492bc73a3103d WatchSource:0}: Error finding container e7fa79f8c5f3306feedd76a1bc5448c0dd7465d551d14c74961492bc73a3103d: Status 404 returned error can't find the container with id e7fa79f8c5f3306feedd76a1bc5448c0dd7465d551d14c74961492bc73a3103d
Sep 18 18:45:43.932782 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:43.932696    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:44.004334 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:44.004301    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:44.562397 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:44.562359    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj" event={"ID":"2e96bcb4-8ab1-4777-93fc-03b873fe4442","Type":"ContainerStarted","Data":"dbece0a4b9560d9abda26865131fbc35c6ee7e4e97d5dba659639e06c7c8e3fd"}
Sep 18 18:45:44.562581 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:44.562423    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"
Sep 18 18:45:44.564150 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:44.564110    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c66559bbb-jdc69" event={"ID":"7ef4bddc-01ec-4ee6-9a48-8f2e81586749","Type":"ContainerStarted","Data":"e7fa79f8c5f3306feedd76a1bc5448c0dd7465d551d14c74961492bc73a3103d"}
Sep 18 18:45:44.580121 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:44.580071    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj" podStartSLOduration=2.167314123 podStartE2EDuration="4.580056813s" totalImagesPullingTime="2.41274269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:44.578127166 +0000 UTC m=+380.776611392" watchObservedRunningTime="2026-09-18 18:45:44.580056813 +0000 UTC m=+380.778541016"
Sep 18 18:45:44.900738 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:44.900638    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:47.582070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:47.582036    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c66559bbb-jdc69" event={"ID":"7ef4bddc-01ec-4ee6-9a48-8f2e81586749","Type":"ContainerStarted","Data":"32d95bd0c723a9d573f8ed49233e4d1bf7a8695d6892fafea528eba5cb890a32"}
Sep 18 18:45:47.600440 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:47.600392    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-c66559bbb-jdc69" podStartSLOduration=2.656989543 podStartE2EDuration="5.600376465s" totalImagesPullingTime="2.943386922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:47.599183033 +0000 UTC m=+383.797667255" watchObservedRunningTime="2026-09-18 18:45:47.600376465 +0000 UTC m=+383.798860669"
Sep 18 18:45:49.186418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:49.186376    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:49.287198 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:49.287157    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:49.394731 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:49.394699    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:49.642063 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:49.642021    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:49.729342 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:49.729306    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:52.952574 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:52.952541    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:53.052139 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:53.052107    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:53.181137 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:53.181102    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:53.273318 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:53.273271    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:53.672474 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:53.672392    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:45:55.572782 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:45:55.572750    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7cddfd44f8-mhnnj"
Sep 18 18:46:01.585909 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.585866    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-868bbdd6c7-jmqkc" podUID="a07e9367-6455-45c3-ba80-ab4765ec5cf8" containerName="console" containerID="cri-o://97026b9ff67026f79174079fedcbd07275af23ff919fceec75c3f2898b38e196" gracePeriod=15
Sep 18 18:46:01.827533 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.827509    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:46:01.916935 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.916853    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a07e9367-6455-45c3-ba80-ab4765ec5cf8-console-oauth-config\") pod \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") "
Sep 18 18:46:01.916935 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.916898    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-oauth-serving-cert\") pod \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") "
Sep 18 18:46:01.917140 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.916961    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjrxt\" (UniqueName: \"kubernetes.io/projected/a07e9367-6455-45c3-ba80-ab4765ec5cf8-kube-api-access-sjrxt\") pod \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") "
Sep 18 18:46:01.917140 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.916981    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-trusted-ca-bundle\") pod \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") "
Sep 18 18:46:01.917140 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.917090    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a07e9367-6455-45c3-ba80-ab4765ec5cf8-console-serving-cert\") pod \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") "
Sep 18 18:46:01.917302 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.917173    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-service-ca\") pod \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") "
Sep 18 18:46:01.917302 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.917199    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-console-config\") pod \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\" (UID: \"a07e9367-6455-45c3-ba80-ab4765ec5cf8\") "
Sep 18 18:46:01.917424 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.917328    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a07e9367-6455-45c3-ba80-ab4765ec5cf8" (UID: "a07e9367-6455-45c3-ba80-ab4765ec5cf8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:46:01.917523 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.917488    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a07e9367-6455-45c3-ba80-ab4765ec5cf8" (UID: "a07e9367-6455-45c3-ba80-ab4765ec5cf8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:46:01.917656 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.917573    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-service-ca" (OuterVolumeSpecName: "service-ca") pod "a07e9367-6455-45c3-ba80-ab4765ec5cf8" (UID: "a07e9367-6455-45c3-ba80-ab4765ec5cf8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:46:01.917731 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.917651    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-console-config" (OuterVolumeSpecName: "console-config") pod "a07e9367-6455-45c3-ba80-ab4765ec5cf8" (UID: "a07e9367-6455-45c3-ba80-ab4765ec5cf8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:46:01.917731 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.917705    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-trusted-ca-bundle\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:46:01.917731 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.917724    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:46:01.917878 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.917739    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-oauth-serving-cert\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:46:01.919157 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.919137    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a07e9367-6455-45c3-ba80-ab4765ec5cf8-kube-api-access-sjrxt" (OuterVolumeSpecName: "kube-api-access-sjrxt") pod "a07e9367-6455-45c3-ba80-ab4765ec5cf8" (UID: "a07e9367-6455-45c3-ba80-ab4765ec5cf8"). InnerVolumeSpecName "kube-api-access-sjrxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:46:01.919221 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.919137    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a07e9367-6455-45c3-ba80-ab4765ec5cf8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a07e9367-6455-45c3-ba80-ab4765ec5cf8" (UID: "a07e9367-6455-45c3-ba80-ab4765ec5cf8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:46:01.919265 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:01.919234    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a07e9367-6455-45c3-ba80-ab4765ec5cf8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a07e9367-6455-45c3-ba80-ab4765ec5cf8" (UID: "a07e9367-6455-45c3-ba80-ab4765ec5cf8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:46:02.018281 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:02.018244    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjrxt\" (UniqueName: \"kubernetes.io/projected/a07e9367-6455-45c3-ba80-ab4765ec5cf8-kube-api-access-sjrxt\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:46:02.018281 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:02.018271    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a07e9367-6455-45c3-ba80-ab4765ec5cf8-console-serving-cert\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:46:02.018281 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:02.018280    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a07e9367-6455-45c3-ba80-ab4765ec5cf8-console-config\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:46:02.018281 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:02.018288    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a07e9367-6455-45c3-ba80-ab4765ec5cf8-console-oauth-config\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:46:02.650424 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:02.650391    2558 generic.go:356] "Generic (PLEG): container finished" podID="a07e9367-6455-45c3-ba80-ab4765ec5cf8" containerID="97026b9ff67026f79174079fedcbd07275af23ff919fceec75c3f2898b38e196" exitCode=0
Sep 18 18:46:02.650860 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:02.650437    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-868bbdd6c7-jmqkc"
Sep 18 18:46:02.650860 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:02.650457    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-868bbdd6c7-jmqkc" event={"ID":"a07e9367-6455-45c3-ba80-ab4765ec5cf8","Type":"ContainerDied","Data":"97026b9ff67026f79174079fedcbd07275af23ff919fceec75c3f2898b38e196"}
Sep 18 18:46:02.650860 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:02.650493    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-868bbdd6c7-jmqkc" event={"ID":"a07e9367-6455-45c3-ba80-ab4765ec5cf8","Type":"ContainerDied","Data":"337373ad43e66f21e9987e1810ab3cf4b5bc9c1c58676c9a5c98041ee3dd8829"}
Sep 18 18:46:02.650860 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:02.650512    2558 scope.go:117] "RemoveContainer" containerID="97026b9ff67026f79174079fedcbd07275af23ff919fceec75c3f2898b38e196"
Sep 18 18:46:02.663399 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:02.663377    2558 scope.go:117] "RemoveContainer" containerID="97026b9ff67026f79174079fedcbd07275af23ff919fceec75c3f2898b38e196"
Sep 18 18:46:02.663700 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:46:02.663676    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97026b9ff67026f79174079fedcbd07275af23ff919fceec75c3f2898b38e196\": container with ID starting with 97026b9ff67026f79174079fedcbd07275af23ff919fceec75c3f2898b38e196 not found: ID does not exist" containerID="97026b9ff67026f79174079fedcbd07275af23ff919fceec75c3f2898b38e196"
Sep 18 18:46:02.663767 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:02.663713    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97026b9ff67026f79174079fedcbd07275af23ff919fceec75c3f2898b38e196"} err="failed to get container status \"97026b9ff67026f79174079fedcbd07275af23ff919fceec75c3f2898b38e196\": rpc error: code = NotFound desc = could not find container \"97026b9ff67026f79174079fedcbd07275af23ff919fceec75c3f2898b38e196\": container with ID starting with 97026b9ff67026f79174079fedcbd07275af23ff919fceec75c3f2898b38e196 not found: ID does not exist"
Sep 18 18:46:02.678293 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:02.678266    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-868bbdd6c7-jmqkc"]
Sep 18 18:46:02.683007 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:02.682985    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-868bbdd6c7-jmqkc"]
Sep 18 18:46:03.856040 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:03.856000    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a07e9367-6455-45c3-ba80-ab4765ec5cf8" path="/var/lib/kubelet/pods/a07e9367-6455-45c3-ba80-ab4765ec5cf8/volumes"
Sep 18 18:46:03.859282 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:03.859251    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:46:03.991031 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:03.990977    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:46:04.113379 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:04.113282    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:46:04.259866 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:04.259814    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:46:05.275180 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:05.275139    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:46:05.397647 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:05.397612    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:46:06.252087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:06.252053    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:46:54.090770 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:54.090697    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:46:54.235051 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:54.234992    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:46:56.178471 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:56.178438    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:46:56.280453 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:56.280425    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:46:58.153015 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:58.152978    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:46:58.226568 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:46:58.226534    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:05.597339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:05.597269    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:09.637113 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:09.637063    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:12.039342 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:12.039289    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:12.426129 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:12.426034    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:14.307035 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:14.306990    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:14.682160 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:14.682072    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:14.865754 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:14.865722    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:16.074057 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:16.074020    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:17.047960 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:17.047928    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:17.483303 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:17.481977    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:17.570465 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:17.570422    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:18.042682 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:18.042646    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:18.342360 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:18.342279    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:18.540564 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:18.540527    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:23.248690 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:23.248652    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:23.345506 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:23.345473    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:26.853522 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:26.853483    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:26.951634 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:26.951598    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:27.947665 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:27.947631    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-87dhq"]
Sep 18 18:47:27.948045 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:27.947986    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a07e9367-6455-45c3-ba80-ab4765ec5cf8" containerName="console"
Sep 18 18:47:27.948045 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:27.947998    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a07e9367-6455-45c3-ba80-ab4765ec5cf8" containerName="console"
Sep 18 18:47:27.948142 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:27.948075    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a07e9367-6455-45c3-ba80-ab4765ec5cf8" containerName="console"
Sep 18 18:47:27.949108 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:27.949078    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:27.949978 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:27.949964    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-87dhq"
Sep 18 18:47:27.952292 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:27.952268    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 18 18:47:27.952427 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:27.952410    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bhpf9\""
Sep 18 18:47:27.952483 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:27.952458    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 18 18:47:27.953452 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:27.953434    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 18 18:47:27.953493 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:27.953434    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 18 18:47:27.959475 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:27.959454    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-87dhq"]
Sep 18 18:47:28.049774 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:28.049743    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbqkl\" (UniqueName: \"kubernetes.io/projected/d2086ce3-6af6-4353-b9ff-2004a8f4ee82-kube-api-access-qbqkl\") pod \"kuadrant-console-plugin-fbdd4f445-87dhq\" (UID: \"d2086ce3-6af6-4353-b9ff-2004a8f4ee82\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-87dhq"
Sep 18 18:47:28.049934 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:28.049791    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2086ce3-6af6-4353-b9ff-2004a8f4ee82-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-87dhq\" (UID: \"d2086ce3-6af6-4353-b9ff-2004a8f4ee82\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-87dhq"
Sep 18 18:47:28.049934 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:28.049902    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d2086ce3-6af6-4353-b9ff-2004a8f4ee82-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-87dhq\" (UID: \"d2086ce3-6af6-4353-b9ff-2004a8f4ee82\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-87dhq"
Sep 18 18:47:28.151321 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:28.151277    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d2086ce3-6af6-4353-b9ff-2004a8f4ee82-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-87dhq\" (UID: \"d2086ce3-6af6-4353-b9ff-2004a8f4ee82\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-87dhq"
Sep 18 18:47:28.151497 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:28.151357    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbqkl\" (UniqueName: \"kubernetes.io/projected/d2086ce3-6af6-4353-b9ff-2004a8f4ee82-kube-api-access-qbqkl\") pod \"kuadrant-console-plugin-fbdd4f445-87dhq\" (UID: \"d2086ce3-6af6-4353-b9ff-2004a8f4ee82\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-87dhq"
Sep 18 18:47:28.151497 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:28.151412    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2086ce3-6af6-4353-b9ff-2004a8f4ee82-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-87dhq\" (UID: \"d2086ce3-6af6-4353-b9ff-2004a8f4ee82\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-87dhq"
Sep 18 18:47:28.151614 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:47:28.151522    2558 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 18 18:47:28.151614 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:47:28.151591    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2086ce3-6af6-4353-b9ff-2004a8f4ee82-plugin-serving-cert podName:d2086ce3-6af6-4353-b9ff-2004a8f4ee82 nodeName:}" failed. No retries permitted until 2026-09-18 18:47:28.651570906 +0000 UTC m=+484.850055088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/d2086ce3-6af6-4353-b9ff-2004a8f4ee82-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-87dhq" (UID: "d2086ce3-6af6-4353-b9ff-2004a8f4ee82") : secret "plugin-serving-cert" not found
Sep 18 18:47:28.151890 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:28.151871    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d2086ce3-6af6-4353-b9ff-2004a8f4ee82-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-87dhq\" (UID: \"d2086ce3-6af6-4353-b9ff-2004a8f4ee82\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-87dhq"
Sep 18 18:47:28.161167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:28.161144    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbqkl\" (UniqueName: \"kubernetes.io/projected/d2086ce3-6af6-4353-b9ff-2004a8f4ee82-kube-api-access-qbqkl\") pod \"kuadrant-console-plugin-fbdd4f445-87dhq\" (UID: \"d2086ce3-6af6-4353-b9ff-2004a8f4ee82\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-87dhq"
Sep 18 18:47:28.416168 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:28.416131    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:28.655644 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:28.655609    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2086ce3-6af6-4353-b9ff-2004a8f4ee82-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-87dhq\" (UID: \"d2086ce3-6af6-4353-b9ff-2004a8f4ee82\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-87dhq"
Sep 18 18:47:28.658054 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:28.658028    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2086ce3-6af6-4353-b9ff-2004a8f4ee82-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-87dhq\" (UID: \"d2086ce3-6af6-4353-b9ff-2004a8f4ee82\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-87dhq"
Sep 18 18:47:28.859195 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:28.859163    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-87dhq"
Sep 18 18:47:29.133348 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:29.133323    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-87dhq"]
Sep 18 18:47:29.136072 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:47:29.136042    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2086ce3_6af6_4353_b9ff_2004a8f4ee82.slice/crio-1439c99cd9682162b00c9d547451423125d6e4dc7938d28a9b1cf6d06c784ed3 WatchSource:0}: Error finding container 1439c99cd9682162b00c9d547451423125d6e4dc7938d28a9b1cf6d06c784ed3: Status 404 returned error can't find the container with id 1439c99cd9682162b00c9d547451423125d6e4dc7938d28a9b1cf6d06c784ed3
Sep 18 18:47:29.324120 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:29.324086    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:29.433264 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:29.433189    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:30.048706 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:30.048661    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-87dhq" event={"ID":"d2086ce3-6af6-4353-b9ff-2004a8f4ee82","Type":"ContainerStarted","Data":"1439c99cd9682162b00c9d547451423125d6e4dc7938d28a9b1cf6d06c784ed3"}
Sep 18 18:47:34.071326 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:34.071285    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-87dhq" event={"ID":"d2086ce3-6af6-4353-b9ff-2004a8f4ee82","Type":"ContainerStarted","Data":"72b87c1ad2a7b852a276c1d6dd045229edd09660c0a0822508f6b13fe6b2e6f9"}
Sep 18 18:47:34.093989 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:34.093944    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-87dhq" podStartSLOduration=2.863485844 podStartE2EDuration="7.093928302s" totalImagesPullingTime="4.230442458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:47:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:47:34.093135712 +0000 UTC m=+490.291619919" watchObservedRunningTime="2026-09-18 18:47:34.093928302 +0000 UTC m=+490.292412510"
Sep 18 18:47:35.787359 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:35.787325    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:35.886396 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:35.886354    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:35.947575 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:35.947541    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:38.177097 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:38.177068    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"]
Sep 18 18:47:38.181269 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:38.181253    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"
Sep 18 18:47:38.184168 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:38.184147    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-99vf4\""
Sep 18 18:47:38.198362 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:38.198336    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"]
Sep 18 18:47:38.342414 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:38.342373    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7d29d19-3ea2-4c50-977f-f4e40f5b4949-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-58ltd\" (UID: \"b7d29d19-3ea2-4c50-977f-f4e40f5b4949\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"
Sep 18 18:47:38.342599 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:38.342476    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kchnl\" (UniqueName: \"kubernetes.io/projected/b7d29d19-3ea2-4c50-977f-f4e40f5b4949-kube-api-access-kchnl\") pod \"kuadrant-operator-controller-manager-5f685c496f-58ltd\" (UID: \"b7d29d19-3ea2-4c50-977f-f4e40f5b4949\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"
Sep 18 18:47:38.443690 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:38.443598    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7d29d19-3ea2-4c50-977f-f4e40f5b4949-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-58ltd\" (UID: \"b7d29d19-3ea2-4c50-977f-f4e40f5b4949\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"
Sep 18 18:47:38.443874 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:38.443692    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kchnl\" (UniqueName: \"kubernetes.io/projected/b7d29d19-3ea2-4c50-977f-f4e40f5b4949-kube-api-access-kchnl\") pod \"kuadrant-operator-controller-manager-5f685c496f-58ltd\" (UID: \"b7d29d19-3ea2-4c50-977f-f4e40f5b4949\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"
Sep 18 18:47:38.444018 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:38.443996    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7d29d19-3ea2-4c50-977f-f4e40f5b4949-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-58ltd\" (UID: \"b7d29d19-3ea2-4c50-977f-f4e40f5b4949\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"
Sep 18 18:47:38.454480 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:38.454459    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kchnl\" (UniqueName: \"kubernetes.io/projected/b7d29d19-3ea2-4c50-977f-f4e40f5b4949-kube-api-access-kchnl\") pod \"kuadrant-operator-controller-manager-5f685c496f-58ltd\" (UID: \"b7d29d19-3ea2-4c50-977f-f4e40f5b4949\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"
Sep 18 18:47:38.491016 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:38.490991    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"
Sep 18 18:47:38.765163 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:38.765133    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"]
Sep 18 18:47:38.768354 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:47:38.768324    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d29d19_3ea2_4c50_977f_f4e40f5b4949.slice/crio-26840faacd6fdb2f796501a5954391244ae18a1b85dfe0eca66350632e0cd1e2 WatchSource:0}: Error finding container 26840faacd6fdb2f796501a5954391244ae18a1b85dfe0eca66350632e0cd1e2: Status 404 returned error can't find the container with id 26840faacd6fdb2f796501a5954391244ae18a1b85dfe0eca66350632e0cd1e2
Sep 18 18:47:38.931455 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:38.931421    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"]
Sep 18 18:47:38.942600 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:38.942573    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"]
Sep 18 18:47:38.953317 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:38.953293    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc"]
Sep 18 18:47:38.957641 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:38.957619    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc"
Sep 18 18:47:38.969696 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:38.969675    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc"]
Sep 18 18:47:39.049260 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:39.049182    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3171060-08f1-4824-91d5-06f781e80eef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-84wzc\" (UID: \"b3171060-08f1-4824-91d5-06f781e80eef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc"
Sep 18 18:47:39.049394 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:39.049324    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5njr\" (UniqueName: \"kubernetes.io/projected/b3171060-08f1-4824-91d5-06f781e80eef-kube-api-access-n5njr\") pod \"kuadrant-operator-controller-manager-5f685c496f-84wzc\" (UID: \"b3171060-08f1-4824-91d5-06f781e80eef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc"
Sep 18 18:47:39.150233 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:39.150202    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5njr\" (UniqueName: \"kubernetes.io/projected/b3171060-08f1-4824-91d5-06f781e80eef-kube-api-access-n5njr\") pod \"kuadrant-operator-controller-manager-5f685c496f-84wzc\" (UID: \"b3171060-08f1-4824-91d5-06f781e80eef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc"
Sep 18 18:47:39.150386 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:39.150262    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3171060-08f1-4824-91d5-06f781e80eef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-84wzc\" (UID: \"b3171060-08f1-4824-91d5-06f781e80eef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc"
Sep 18 18:47:39.150659 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:39.150643    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3171060-08f1-4824-91d5-06f781e80eef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-84wzc\" (UID: \"b3171060-08f1-4824-91d5-06f781e80eef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc"
Sep 18 18:47:39.164042 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:39.164011    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5njr\" (UniqueName: \"kubernetes.io/projected/b3171060-08f1-4824-91d5-06f781e80eef-kube-api-access-n5njr\") pod \"kuadrant-operator-controller-manager-5f685c496f-84wzc\" (UID: \"b3171060-08f1-4824-91d5-06f781e80eef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc"
Sep 18 18:47:39.269145 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:39.269117    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc"
Sep 18 18:47:39.554058 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:39.554033    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc"]
Sep 18 18:47:39.556488 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:47:39.556463    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3171060_08f1_4824_91d5_06f781e80eef.slice/crio-f36f064105b9c65ce6a8e4658464fd4b2f53a1c83ea574350f3192b85d9dcb46 WatchSource:0}: Error finding container f36f064105b9c65ce6a8e4658464fd4b2f53a1c83ea574350f3192b85d9dcb46: Status 404 returned error can't find the container with id f36f064105b9c65ce6a8e4658464fd4b2f53a1c83ea574350f3192b85d9dcb46
Sep 18 18:47:40.102242 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:40.102203    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc" event={"ID":"b3171060-08f1-4824-91d5-06f781e80eef","Type":"ContainerStarted","Data":"f36f064105b9c65ce6a8e4658464fd4b2f53a1c83ea574350f3192b85d9dcb46"}
Sep 18 18:47:42.460452 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:42.460387    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:42.596442 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:42.596405    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:44.125996 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:44.125957    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc" event={"ID":"b3171060-08f1-4824-91d5-06f781e80eef","Type":"ContainerStarted","Data":"8f3d802d97f52e95b285881667badb94a2bb0efe8b43e8f15d9a33137d18aca1"}
Sep 18 18:47:44.126459 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:44.126017    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc"
Sep 18 18:47:44.127638 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:44.127613    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd" podUID="b7d29d19-3ea2-4c50-977f-f4e40f5b4949" containerName="manager" containerID="cri-o://f0c7e7bd519240159ff4d816893ab70d51452e7ec82d1de005c35c0dcace4a23" gracePeriod=2
Sep 18 18:47:44.150909 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:44.150591    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc" podStartSLOduration=2.37582658 podStartE2EDuration="6.150573363s" totalImagesPullingTime="3.774746783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:47:44.14906625 +0000 UTC m=+500.347550457" watchObservedRunningTime="2026-09-18 18:47:44.150573363 +0000 UTC m=+500.349057571"
Sep 18 18:47:44.371278 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:44.371257    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"
Sep 18 18:47:44.373470 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:47:44.373448    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-58ltd\" is forbidden: User \"system:node:ip-10-0-130-118.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-118.ec2.internal' and this object" podUID="b7d29d19-3ea2-4c50-977f-f4e40f5b4949" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"
Sep 18 18:47:44.501110 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:44.501091    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kchnl\" (UniqueName: \"kubernetes.io/projected/b7d29d19-3ea2-4c50-977f-f4e40f5b4949-kube-api-access-kchnl\") pod \"b7d29d19-3ea2-4c50-977f-f4e40f5b4949\" (UID: \"b7d29d19-3ea2-4c50-977f-f4e40f5b4949\") "
Sep 18 18:47:44.501229 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:44.501159    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7d29d19-3ea2-4c50-977f-f4e40f5b4949-extensions-socket-volume\") pod \"b7d29d19-3ea2-4c50-977f-f4e40f5b4949\" (UID: \"b7d29d19-3ea2-4c50-977f-f4e40f5b4949\") "
Sep 18 18:47:44.501371 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:44.501350    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d29d19-3ea2-4c50-977f-f4e40f5b4949-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b7d29d19-3ea2-4c50-977f-f4e40f5b4949" (UID: "b7d29d19-3ea2-4c50-977f-f4e40f5b4949"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:47:44.503267 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:44.503246    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7d29d19-3ea2-4c50-977f-f4e40f5b4949-kube-api-access-kchnl" (OuterVolumeSpecName: "kube-api-access-kchnl") pod "b7d29d19-3ea2-4c50-977f-f4e40f5b4949" (UID: "b7d29d19-3ea2-4c50-977f-f4e40f5b4949"). InnerVolumeSpecName "kube-api-access-kchnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:47:44.602393 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:44.602367    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7d29d19-3ea2-4c50-977f-f4e40f5b4949-extensions-socket-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:47:44.602393 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:44.602388    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kchnl\" (UniqueName: \"kubernetes.io/projected/b7d29d19-3ea2-4c50-977f-f4e40f5b4949-kube-api-access-kchnl\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:47:45.133511 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:45.133479    2558 generic.go:356] "Generic (PLEG): container finished" podID="b7d29d19-3ea2-4c50-977f-f4e40f5b4949" containerID="f0c7e7bd519240159ff4d816893ab70d51452e7ec82d1de005c35c0dcace4a23" exitCode=2
Sep 18 18:47:45.133956 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:45.133526    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"
Sep 18 18:47:45.133956 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:45.133579    2558 scope.go:117] "RemoveContainer" containerID="f0c7e7bd519240159ff4d816893ab70d51452e7ec82d1de005c35c0dcace4a23"
Sep 18 18:47:45.135910 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:47:45.135798    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-58ltd\" is forbidden: User \"system:node:ip-10-0-130-118.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-118.ec2.internal' and this object" podUID="b7d29d19-3ea2-4c50-977f-f4e40f5b4949" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"
Sep 18 18:47:45.142624 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:45.142611    2558 scope.go:117] "RemoveContainer" containerID="f0c7e7bd519240159ff4d816893ab70d51452e7ec82d1de005c35c0dcace4a23"
Sep 18 18:47:45.142947 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:47:45.142931    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0c7e7bd519240159ff4d816893ab70d51452e7ec82d1de005c35c0dcace4a23\": container with ID starting with f0c7e7bd519240159ff4d816893ab70d51452e7ec82d1de005c35c0dcace4a23 not found: ID does not exist" containerID="f0c7e7bd519240159ff4d816893ab70d51452e7ec82d1de005c35c0dcace4a23"
Sep 18 18:47:45.142995 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:45.142955    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0c7e7bd519240159ff4d816893ab70d51452e7ec82d1de005c35c0dcace4a23"} err="failed to get container status \"f0c7e7bd519240159ff4d816893ab70d51452e7ec82d1de005c35c0dcace4a23\": rpc error: code = NotFound desc = could not find container \"f0c7e7bd519240159ff4d816893ab70d51452e7ec82d1de005c35c0dcace4a23\": container with ID starting with f0c7e7bd519240159ff4d816893ab70d51452e7ec82d1de005c35c0dcace4a23 not found: ID does not exist"
Sep 18 18:47:45.144298 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:47:45.144279    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-58ltd\" is forbidden: User \"system:node:ip-10-0-130-118.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-118.ec2.internal' and this object" podUID="b7d29d19-3ea2-4c50-977f-f4e40f5b4949" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"
Sep 18 18:47:45.842480 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:47:45.842448    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-58ltd\" is forbidden: User \"system:node:ip-10-0-130-118.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-118.ec2.internal' and this object" podUID="b7d29d19-3ea2-4c50-977f-f4e40f5b4949" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-58ltd"
Sep 18 18:47:45.844386 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:45.844364    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7d29d19-3ea2-4c50-977f-f4e40f5b4949" path="/var/lib/kubelet/pods/b7d29d19-3ea2-4c50-977f-f4e40f5b4949/volumes"
Sep 18 18:47:46.250344 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:46.250311    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:46.350385 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:46.350359    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:46.456301 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:46.456273    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:46.556844 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:46.556759    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:50.252947 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:50.252912    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:50.352970 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:50.352946    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:55.135519 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:55.135494    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc"
Sep 18 18:47:56.030860 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:56.030810    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:56.133557 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:56.133522    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:56.253450 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:56.253417    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:47:56.353833 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:47:56.353739    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:48:07.205704 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:07.205623    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc"]
Sep 18 18:48:07.206210 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:07.205869    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc" podUID="b3171060-08f1-4824-91d5-06f781e80eef" containerName="manager" containerID="cri-o://8f3d802d97f52e95b285881667badb94a2bb0efe8b43e8f15d9a33137d18aca1" gracePeriod=10
Sep 18 18:48:07.452143 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:07.452123    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc"
Sep 18 18:48:07.518378 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:07.518349    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5njr\" (UniqueName: \"kubernetes.io/projected/b3171060-08f1-4824-91d5-06f781e80eef-kube-api-access-n5njr\") pod \"b3171060-08f1-4824-91d5-06f781e80eef\" (UID: \"b3171060-08f1-4824-91d5-06f781e80eef\") "
Sep 18 18:48:07.518525 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:07.518459    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3171060-08f1-4824-91d5-06f781e80eef-extensions-socket-volume\") pod \"b3171060-08f1-4824-91d5-06f781e80eef\" (UID: \"b3171060-08f1-4824-91d5-06f781e80eef\") "
Sep 18 18:48:07.518916 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:07.518887    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3171060-08f1-4824-91d5-06f781e80eef-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b3171060-08f1-4824-91d5-06f781e80eef" (UID: "b3171060-08f1-4824-91d5-06f781e80eef"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:48:07.520319 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:07.520293    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3171060-08f1-4824-91d5-06f781e80eef-kube-api-access-n5njr" (OuterVolumeSpecName: "kube-api-access-n5njr") pod "b3171060-08f1-4824-91d5-06f781e80eef" (UID: "b3171060-08f1-4824-91d5-06f781e80eef"). InnerVolumeSpecName "kube-api-access-n5njr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:48:07.619505 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:07.619478    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3171060-08f1-4824-91d5-06f781e80eef-extensions-socket-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:48:07.619505 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:07.619502    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5njr\" (UniqueName: \"kubernetes.io/projected/b3171060-08f1-4824-91d5-06f781e80eef-kube-api-access-n5njr\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:48:08.247547 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:08.247516    2558 generic.go:356] "Generic (PLEG): container finished" podID="b3171060-08f1-4824-91d5-06f781e80eef" containerID="8f3d802d97f52e95b285881667badb94a2bb0efe8b43e8f15d9a33137d18aca1" exitCode=0
Sep 18 18:48:08.247963 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:08.247565    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc" event={"ID":"b3171060-08f1-4824-91d5-06f781e80eef","Type":"ContainerDied","Data":"8f3d802d97f52e95b285881667badb94a2bb0efe8b43e8f15d9a33137d18aca1"}
Sep 18 18:48:08.247963 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:08.247573    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc"
Sep 18 18:48:08.247963 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:08.247613    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc" event={"ID":"b3171060-08f1-4824-91d5-06f781e80eef","Type":"ContainerDied","Data":"f36f064105b9c65ce6a8e4658464fd4b2f53a1c83ea574350f3192b85d9dcb46"}
Sep 18 18:48:08.247963 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:08.247637    2558 scope.go:117] "RemoveContainer" containerID="8f3d802d97f52e95b285881667badb94a2bb0efe8b43e8f15d9a33137d18aca1"
Sep 18 18:48:08.255889 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:08.255867    2558 scope.go:117] "RemoveContainer" containerID="8f3d802d97f52e95b285881667badb94a2bb0efe8b43e8f15d9a33137d18aca1"
Sep 18 18:48:08.256142 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:48:08.256123    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f3d802d97f52e95b285881667badb94a2bb0efe8b43e8f15d9a33137d18aca1\": container with ID starting with 8f3d802d97f52e95b285881667badb94a2bb0efe8b43e8f15d9a33137d18aca1 not found: ID does not exist" containerID="8f3d802d97f52e95b285881667badb94a2bb0efe8b43e8f15d9a33137d18aca1"
Sep 18 18:48:08.256206 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:08.256150    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f3d802d97f52e95b285881667badb94a2bb0efe8b43e8f15d9a33137d18aca1"} err="failed to get container status \"8f3d802d97f52e95b285881667badb94a2bb0efe8b43e8f15d9a33137d18aca1\": rpc error: code = NotFound desc = could not find container \"8f3d802d97f52e95b285881667badb94a2bb0efe8b43e8f15d9a33137d18aca1\": container with ID starting with 8f3d802d97f52e95b285881667badb94a2bb0efe8b43e8f15d9a33137d18aca1 not found: ID does not exist"
Sep 18 18:48:08.265342 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:08.265317    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc"]
Sep 18 18:48:08.273305 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:08.273279    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-84wzc"]
Sep 18 18:48:08.660180 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:08.660090    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:48:09.844498 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:09.844465    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3171060-08f1-4824-91d5-06f781e80eef" path="/var/lib/kubelet/pods/b3171060-08f1-4824-91d5-06f781e80eef/volumes"
Sep 18 18:48:11.489554 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:11.489508    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:48:11.680114 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:11.680082    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:48:24.595431 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.595397    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-8ff6v"]
Sep 18 18:48:24.595784 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.595748    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3171060-08f1-4824-91d5-06f781e80eef" containerName="manager"
Sep 18 18:48:24.595784 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.595760    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3171060-08f1-4824-91d5-06f781e80eef" containerName="manager"
Sep 18 18:48:24.595784 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.595779    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7d29d19-3ea2-4c50-977f-f4e40f5b4949" containerName="manager"
Sep 18 18:48:24.595784 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.595785    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7d29d19-3ea2-4c50-977f-f4e40f5b4949" containerName="manager"
Sep 18 18:48:24.595953 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.595891    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3171060-08f1-4824-91d5-06f781e80eef" containerName="manager"
Sep 18 18:48:24.595953 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.595901    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7d29d19-3ea2-4c50-977f-f4e40f5b4949" containerName="manager"
Sep 18 18:48:24.599956 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.599940    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-8ff6v"
Sep 18 18:48:24.602562 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.602537    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 18 18:48:24.604121 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.604098    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:48:24.608062 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.608040    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-8ff6v"]
Sep 18 18:48:24.695316 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.695283    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-8ff6v"]
Sep 18 18:48:24.697715 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.697684    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:48:24.758556 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.758522    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:48:24.761417 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.761394    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7pw7\" (UniqueName: \"kubernetes.io/projected/f644bb30-a052-4809-95e7-1439b214c4e3-kube-api-access-v7pw7\") pod \"limitador-limitador-8588749977-8ff6v\" (UID: \"f644bb30-a052-4809-95e7-1439b214c4e3\") " pod="kuadrant-system/limitador-limitador-8588749977-8ff6v"
Sep 18 18:48:24.761540 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.761429    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f644bb30-a052-4809-95e7-1439b214c4e3-config-file\") pod \"limitador-limitador-8588749977-8ff6v\" (UID: \"f644bb30-a052-4809-95e7-1439b214c4e3\") " pod="kuadrant-system/limitador-limitador-8588749977-8ff6v"
Sep 18 18:48:24.862610 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.862534    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7pw7\" (UniqueName: \"kubernetes.io/projected/f644bb30-a052-4809-95e7-1439b214c4e3-kube-api-access-v7pw7\") pod \"limitador-limitador-8588749977-8ff6v\" (UID: \"f644bb30-a052-4809-95e7-1439b214c4e3\") " pod="kuadrant-system/limitador-limitador-8588749977-8ff6v"
Sep 18 18:48:24.862610 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.862565    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f644bb30-a052-4809-95e7-1439b214c4e3-config-file\") pod \"limitador-limitador-8588749977-8ff6v\" (UID: \"f644bb30-a052-4809-95e7-1439b214c4e3\") " pod="kuadrant-system/limitador-limitador-8588749977-8ff6v"
Sep 18 18:48:24.863153 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.863134    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f644bb30-a052-4809-95e7-1439b214c4e3-config-file\") pod \"limitador-limitador-8588749977-8ff6v\" (UID: \"f644bb30-a052-4809-95e7-1439b214c4e3\") " pod="kuadrant-system/limitador-limitador-8588749977-8ff6v"
Sep 18 18:48:24.872108 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.872091    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7pw7\" (UniqueName: \"kubernetes.io/projected/f644bb30-a052-4809-95e7-1439b214c4e3-kube-api-access-v7pw7\") pod \"limitador-limitador-8588749977-8ff6v\" (UID: \"f644bb30-a052-4809-95e7-1439b214c4e3\") " pod="kuadrant-system/limitador-limitador-8588749977-8ff6v"
Sep 18 18:48:24.911660 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:24.911636    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-8ff6v"
Sep 18 18:48:25.180010 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:25.179986    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-8ff6v"]
Sep 18 18:48:25.182621 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:48:25.182597    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf644bb30_a052_4809_95e7_1439b214c4e3.slice/crio-a5c55c0fb837f817c7afee8b21f37eb1560c3fb66c8080196837e9d4ab084348 WatchSource:0}: Error finding container a5c55c0fb837f817c7afee8b21f37eb1560c3fb66c8080196837e9d4ab084348: Status 404 returned error can't find the container with id a5c55c0fb837f817c7afee8b21f37eb1560c3fb66c8080196837e9d4ab084348
Sep 18 18:48:25.332454 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:25.332423    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-8ff6v" event={"ID":"f644bb30-a052-4809-95e7-1439b214c4e3","Type":"ContainerStarted","Data":"a5c55c0fb837f817c7afee8b21f37eb1560c3fb66c8080196837e9d4ab084348"}
Sep 18 18:48:25.432300 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:25.432211    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:48:25.490706 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:25.490673    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:48:25.571164 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:25.571136    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-mwvm4"]
Sep 18 18:48:25.574116 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:25.574099    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-mwvm4"
Sep 18 18:48:25.579649 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:25.579630    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6xsp7\""
Sep 18 18:48:25.589505 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:25.589480    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mwvm4"]
Sep 18 18:48:25.671021 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:25.670994    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s56np\" (UniqueName: \"kubernetes.io/projected/7882777d-7c40-47c4-96c0-5db197814bb8-kube-api-access-s56np\") pod \"authorino-59c974bd48-mwvm4\" (UID: \"7882777d-7c40-47c4-96c0-5db197814bb8\") " pod="kuadrant-system/authorino-59c974bd48-mwvm4"
Sep 18 18:48:25.771804 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:25.771775    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s56np\" (UniqueName: \"kubernetes.io/projected/7882777d-7c40-47c4-96c0-5db197814bb8-kube-api-access-s56np\") pod \"authorino-59c974bd48-mwvm4\" (UID: \"7882777d-7c40-47c4-96c0-5db197814bb8\") " pod="kuadrant-system/authorino-59c974bd48-mwvm4"
Sep 18 18:48:25.780312 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:25.780271    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s56np\" (UniqueName: \"kubernetes.io/projected/7882777d-7c40-47c4-96c0-5db197814bb8-kube-api-access-s56np\") pod \"authorino-59c974bd48-mwvm4\" (UID: \"7882777d-7c40-47c4-96c0-5db197814bb8\") " pod="kuadrant-system/authorino-59c974bd48-mwvm4"
Sep 18 18:48:25.886177 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:25.886136    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6xsp7\""
Sep 18 18:48:25.894462 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:25.894431    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-mwvm4"
Sep 18 18:48:26.171385 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:26.171151    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mwvm4"]
Sep 18 18:48:26.237631 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:48:26.237599    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7882777d_7c40_47c4_96c0_5db197814bb8.slice/crio-10b2e6fd260445d571991713e240f58451ce370976213f50247da3289afeb091 WatchSource:0}: Error finding container 10b2e6fd260445d571991713e240f58451ce370976213f50247da3289afeb091: Status 404 returned error can't find the container with id 10b2e6fd260445d571991713e240f58451ce370976213f50247da3289afeb091
Sep 18 18:48:26.339202 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:26.339172    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-8ff6v" event={"ID":"f644bb30-a052-4809-95e7-1439b214c4e3","Type":"ContainerStarted","Data":"8b0a992f714ae65d197e45730706d00a03af71802f0803e120e74714b6fc81f6"}
Sep 18 18:48:26.339321 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:26.339290    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-8ff6v"
Sep 18 18:48:26.340398 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:26.340377    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-mwvm4" event={"ID":"7882777d-7c40-47c4-96c0-5db197814bb8","Type":"ContainerStarted","Data":"10b2e6fd260445d571991713e240f58451ce370976213f50247da3289afeb091"}
Sep 18 18:48:26.358077 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:26.358035    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-8ff6v" podStartSLOduration=1.279473872 podStartE2EDuration="2.35802213s" totalImagesPullingTime="1.078548258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:48:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:48:26.356841701 +0000 UTC m=+542.555325931" watchObservedRunningTime="2026-09-18 18:48:26.35802213 +0000 UTC m=+542.556506334"
Sep 18 18:48:29.358849 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:29.358798    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-mwvm4" event={"ID":"7882777d-7c40-47c4-96c0-5db197814bb8","Type":"ContainerStarted","Data":"ce0356dc7297e2cdfec3a2b1af913c67aa77b6f03fda6c66226099c2539419ea"}
Sep 18 18:48:29.376282 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:29.376237    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-mwvm4" podStartSLOduration=2.211832499 podStartE2EDuration="4.376221015s" totalImagesPullingTime="2.164388516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:48:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:48:29.375391207 +0000 UTC m=+545.573875415" watchObservedRunningTime="2026-09-18 18:48:29.376221015 +0000 UTC m=+545.574705219"
Sep 18 18:48:34.013252 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:34.013215    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:48:37.348074 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:37.348043    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-8ff6v"
Sep 18 18:48:37.735285 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:37.735207    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-8ff6v"]
Sep 18 18:48:37.735437 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:37.735385    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-8ff6v" podUID="f644bb30-a052-4809-95e7-1439b214c4e3" containerName="limitador" containerID="cri-o://8b0a992f714ae65d197e45730706d00a03af71802f0803e120e74714b6fc81f6" gracePeriod=30
Sep 18 18:48:38.405532 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:38.405436    2558 generic.go:356] "Generic (PLEG): container finished" podID="f644bb30-a052-4809-95e7-1439b214c4e3" containerID="8b0a992f714ae65d197e45730706d00a03af71802f0803e120e74714b6fc81f6" exitCode=0
Sep 18 18:48:38.405532 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:38.405510    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-8ff6v" event={"ID":"f644bb30-a052-4809-95e7-1439b214c4e3","Type":"ContainerDied","Data":"8b0a992f714ae65d197e45730706d00a03af71802f0803e120e74714b6fc81f6"}
Sep 18 18:48:38.678613 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:38.678586    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-8ff6v"
Sep 18 18:48:38.770843 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:38.770789    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f644bb30-a052-4809-95e7-1439b214c4e3-config-file\") pod \"f644bb30-a052-4809-95e7-1439b214c4e3\" (UID: \"f644bb30-a052-4809-95e7-1439b214c4e3\") "
Sep 18 18:48:38.771028 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:38.771009    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7pw7\" (UniqueName: \"kubernetes.io/projected/f644bb30-a052-4809-95e7-1439b214c4e3-kube-api-access-v7pw7\") pod \"f644bb30-a052-4809-95e7-1439b214c4e3\" (UID: \"f644bb30-a052-4809-95e7-1439b214c4e3\") "
Sep 18 18:48:38.771233 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:38.771212    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f644bb30-a052-4809-95e7-1439b214c4e3-config-file" (OuterVolumeSpecName: "config-file") pod "f644bb30-a052-4809-95e7-1439b214c4e3" (UID: "f644bb30-a052-4809-95e7-1439b214c4e3"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:48:38.771305 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:38.771291    2558 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f644bb30-a052-4809-95e7-1439b214c4e3-config-file\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:48:38.773035 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:38.773009    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f644bb30-a052-4809-95e7-1439b214c4e3-kube-api-access-v7pw7" (OuterVolumeSpecName: "kube-api-access-v7pw7") pod "f644bb30-a052-4809-95e7-1439b214c4e3" (UID: "f644bb30-a052-4809-95e7-1439b214c4e3"). InnerVolumeSpecName "kube-api-access-v7pw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:48:38.871737 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:38.871693    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7pw7\" (UniqueName: \"kubernetes.io/projected/f644bb30-a052-4809-95e7-1439b214c4e3-kube-api-access-v7pw7\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:48:39.411404 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:39.411376    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-8ff6v"
Sep 18 18:48:39.411404 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:39.411394    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-8ff6v" event={"ID":"f644bb30-a052-4809-95e7-1439b214c4e3","Type":"ContainerDied","Data":"a5c55c0fb837f817c7afee8b21f37eb1560c3fb66c8080196837e9d4ab084348"}
Sep 18 18:48:39.411853 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:39.411447    2558 scope.go:117] "RemoveContainer" containerID="8b0a992f714ae65d197e45730706d00a03af71802f0803e120e74714b6fc81f6"
Sep 18 18:48:39.435380 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:39.435358    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-8ff6v"]
Sep 18 18:48:39.439188 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:39.439167    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-8ff6v"]
Sep 18 18:48:39.844175 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:39.844137    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f644bb30-a052-4809-95e7-1439b214c4e3" path="/var/lib/kubelet/pods/f644bb30-a052-4809-95e7-1439b214c4e3/volumes"
Sep 18 18:48:42.942225 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:42.942191    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:48:43.044406 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.044369    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:48:43.359023 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.358990    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9ppsg"]
Sep 18 18:48:43.359438 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.359421    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f644bb30-a052-4809-95e7-1439b214c4e3" containerName="limitador"
Sep 18 18:48:43.359438 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.359439    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f644bb30-a052-4809-95e7-1439b214c4e3" containerName="limitador"
Sep 18 18:48:43.359545 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.359492    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f644bb30-a052-4809-95e7-1439b214c4e3" containerName="limitador"
Sep 18 18:48:43.363567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.363551    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9ppsg"
Sep 18 18:48:43.366066 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.366044    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:48:43.366180 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.366072    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-znshr\""
Sep 18 18:48:43.366973 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.366958    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 18 18:48:43.367052 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.366958    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 18:48:43.371003 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.370979    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9ppsg"]
Sep 18 18:48:43.387613 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.387589    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:48:43.409659 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.409639    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2ee3eb35-9bf5-494c-8dfd-d5b4291093cb-data\") pod \"postgres-69fb4594cf-9ppsg\" (UID: \"2ee3eb35-9bf5-494c-8dfd-d5b4291093cb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9ppsg"
Sep 18 18:48:43.409757 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.409675    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cq8j\" (UniqueName: \"kubernetes.io/projected/2ee3eb35-9bf5-494c-8dfd-d5b4291093cb-kube-api-access-6cq8j\") pod \"postgres-69fb4594cf-9ppsg\" (UID: \"2ee3eb35-9bf5-494c-8dfd-d5b4291093cb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9ppsg"
Sep 18 18:48:43.510246 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.510218    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2ee3eb35-9bf5-494c-8dfd-d5b4291093cb-data\") pod \"postgres-69fb4594cf-9ppsg\" (UID: \"2ee3eb35-9bf5-494c-8dfd-d5b4291093cb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9ppsg"
Sep 18 18:48:43.510412 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.510258    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cq8j\" (UniqueName: \"kubernetes.io/projected/2ee3eb35-9bf5-494c-8dfd-d5b4291093cb-kube-api-access-6cq8j\") pod \"postgres-69fb4594cf-9ppsg\" (UID: \"2ee3eb35-9bf5-494c-8dfd-d5b4291093cb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9ppsg"
Sep 18 18:48:43.510626 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.510608    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2ee3eb35-9bf5-494c-8dfd-d5b4291093cb-data\") pod \"postgres-69fb4594cf-9ppsg\" (UID: \"2ee3eb35-9bf5-494c-8dfd-d5b4291093cb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9ppsg"
Sep 18 18:48:43.522385 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.522359    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cq8j\" (UniqueName: \"kubernetes.io/projected/2ee3eb35-9bf5-494c-8dfd-d5b4291093cb-kube-api-access-6cq8j\") pod \"postgres-69fb4594cf-9ppsg\" (UID: \"2ee3eb35-9bf5-494c-8dfd-d5b4291093cb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9ppsg"
Sep 18 18:48:43.675108 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.675024    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9ppsg"
Sep 18 18:48:43.943780 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:43.943753    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9ppsg"]
Sep 18 18:48:43.945288 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:48:43.945249    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ee3eb35_9bf5_494c_8dfd_d5b4291093cb.slice/crio-6fbb9bdf31b0871b8c95c83d2c2cd42bff97f4e378a3e1490ee339be362a774e WatchSource:0}: Error finding container 6fbb9bdf31b0871b8c95c83d2c2cd42bff97f4e378a3e1490ee339be362a774e: Status 404 returned error can't find the container with id 6fbb9bdf31b0871b8c95c83d2c2cd42bff97f4e378a3e1490ee339be362a774e
Sep 18 18:48:44.441533 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:44.441499    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9ppsg" event={"ID":"2ee3eb35-9bf5-494c-8dfd-d5b4291093cb","Type":"ContainerStarted","Data":"6fbb9bdf31b0871b8c95c83d2c2cd42bff97f4e378a3e1490ee339be362a774e"}
Sep 18 18:48:49.471520 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:49.471485    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9ppsg" event={"ID":"2ee3eb35-9bf5-494c-8dfd-d5b4291093cb","Type":"ContainerStarted","Data":"cb6035c67e0db8062b58be178c28ed90cf43f7d146452ef35e76a698c9170884"}
Sep 18 18:48:49.472031 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:49.471554    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9ppsg"
Sep 18 18:48:49.489630 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:49.489585    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9ppsg" podStartSLOduration=1.547245562 podStartE2EDuration="6.48957154s" totalImagesPullingTime="4.942325978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:48:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:48:49.487773044 +0000 UTC m=+565.686257268" watchObservedRunningTime="2026-09-18 18:48:49.48957154 +0000 UTC m=+565.688055744"
Sep 18 18:48:55.505506 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:55.505475    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9ppsg"
Sep 18 18:48:55.797852 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:55.797705    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:48:55.897495 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:55.897462    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:48:57.536089 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:57.536059    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mwvm4"]
Sep 18 18:48:57.536503 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:57.536219    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-mwvm4" podUID="7882777d-7c40-47c4-96c0-5db197814bb8" containerName="authorino" containerID="cri-o://ce0356dc7297e2cdfec3a2b1af913c67aa77b6f03fda6c66226099c2539419ea" gracePeriod=30
Sep 18 18:48:57.780274 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:57.780251    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-mwvm4"
Sep 18 18:48:57.841515 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:57.841434    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s56np\" (UniqueName: \"kubernetes.io/projected/7882777d-7c40-47c4-96c0-5db197814bb8-kube-api-access-s56np\") pod \"7882777d-7c40-47c4-96c0-5db197814bb8\" (UID: \"7882777d-7c40-47c4-96c0-5db197814bb8\") "
Sep 18 18:48:57.843519 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:57.843492    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7882777d-7c40-47c4-96c0-5db197814bb8-kube-api-access-s56np" (OuterVolumeSpecName: "kube-api-access-s56np") pod "7882777d-7c40-47c4-96c0-5db197814bb8" (UID: "7882777d-7c40-47c4-96c0-5db197814bb8"). InnerVolumeSpecName "kube-api-access-s56np". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:48:57.942973 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:57.942952    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s56np\" (UniqueName: \"kubernetes.io/projected/7882777d-7c40-47c4-96c0-5db197814bb8-kube-api-access-s56np\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:48:58.515283 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:58.515252    2558 generic.go:356] "Generic (PLEG): container finished" podID="7882777d-7c40-47c4-96c0-5db197814bb8" containerID="ce0356dc7297e2cdfec3a2b1af913c67aa77b6f03fda6c66226099c2539419ea" exitCode=0
Sep 18 18:48:58.515463 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:58.515291    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-mwvm4" event={"ID":"7882777d-7c40-47c4-96c0-5db197814bb8","Type":"ContainerDied","Data":"ce0356dc7297e2cdfec3a2b1af913c67aa77b6f03fda6c66226099c2539419ea"}
Sep 18 18:48:58.515463 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:58.515320    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-mwvm4" event={"ID":"7882777d-7c40-47c4-96c0-5db197814bb8","Type":"ContainerDied","Data":"10b2e6fd260445d571991713e240f58451ce370976213f50247da3289afeb091"}
Sep 18 18:48:58.515463 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:58.515293    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-mwvm4"
Sep 18 18:48:58.515463 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:58.515337    2558 scope.go:117] "RemoveContainer" containerID="ce0356dc7297e2cdfec3a2b1af913c67aa77b6f03fda6c66226099c2539419ea"
Sep 18 18:48:58.523628 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:58.523612    2558 scope.go:117] "RemoveContainer" containerID="ce0356dc7297e2cdfec3a2b1af913c67aa77b6f03fda6c66226099c2539419ea"
Sep 18 18:48:58.523901 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:48:58.523885    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce0356dc7297e2cdfec3a2b1af913c67aa77b6f03fda6c66226099c2539419ea\": container with ID starting with ce0356dc7297e2cdfec3a2b1af913c67aa77b6f03fda6c66226099c2539419ea not found: ID does not exist" containerID="ce0356dc7297e2cdfec3a2b1af913c67aa77b6f03fda6c66226099c2539419ea"
Sep 18 18:48:58.523961 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:58.523909    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce0356dc7297e2cdfec3a2b1af913c67aa77b6f03fda6c66226099c2539419ea"} err="failed to get container status \"ce0356dc7297e2cdfec3a2b1af913c67aa77b6f03fda6c66226099c2539419ea\": rpc error: code = NotFound desc = could not find container \"ce0356dc7297e2cdfec3a2b1af913c67aa77b6f03fda6c66226099c2539419ea\": container with ID starting with ce0356dc7297e2cdfec3a2b1af913c67aa77b6f03fda6c66226099c2539419ea not found: ID does not exist"
Sep 18 18:48:58.531489 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:58.531466    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mwvm4"]
Sep 18 18:48:58.533962 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:58.533942    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mwvm4"]
Sep 18 18:48:59.844121 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:48:59.844088    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7882777d-7c40-47c4-96c0-5db197814bb8" path="/var/lib/kubelet/pods/7882777d-7c40-47c4-96c0-5db197814bb8/volumes"
Sep 18 18:49:02.032861 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.032812    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:49:02.111398 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.111362    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:49:02.115655 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.115633    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-fx58b"]
Sep 18 18:49:02.116055 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.116040    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7882777d-7c40-47c4-96c0-5db197814bb8" containerName="authorino"
Sep 18 18:49:02.116148 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.116058    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7882777d-7c40-47c4-96c0-5db197814bb8" containerName="authorino"
Sep 18 18:49:02.116200 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.116168    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7882777d-7c40-47c4-96c0-5db197814bb8" containerName="authorino"
Sep 18 18:49:02.172368 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.172333    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:49:02.215032 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.215001    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-fx58b"]
Sep 18 18:49:02.215166 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.215116    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-fx58b"
Sep 18 18:49:02.217741 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.217724    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 18 18:49:02.218831 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.218799    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-x7kbw\""
Sep 18 18:49:02.219056 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.219035    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 18 18:49:02.219141 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.219078    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 18 18:49:02.226506 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.226484    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:49:02.281028 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.281002    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64fbd675b6-m524m"]
Sep 18 18:49:02.285675 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.285623    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79502e36-ec13-40e1-8510-82fdef1c0573-webhook-cert\") pod \"maas-controller-75dc6cbdf-fx58b\" (UID: \"79502e36-ec13-40e1-8510-82fdef1c0573\") " pod="opendatahub/maas-controller-75dc6cbdf-fx58b"
Sep 18 18:49:02.285675 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.285659    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79502e36-ec13-40e1-8510-82fdef1c0573-metrics-certs\") pod \"maas-controller-75dc6cbdf-fx58b\" (UID: \"79502e36-ec13-40e1-8510-82fdef1c0573\") " pod="opendatahub/maas-controller-75dc6cbdf-fx58b"
Sep 18 18:49:02.285797 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.285748    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcvp5\" (UniqueName: \"kubernetes.io/projected/79502e36-ec13-40e1-8510-82fdef1c0573-kube-api-access-jcvp5\") pod \"maas-controller-75dc6cbdf-fx58b\" (UID: \"79502e36-ec13-40e1-8510-82fdef1c0573\") " pod="opendatahub/maas-controller-75dc6cbdf-fx58b"
Sep 18 18:49:02.301404 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.301367    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64fbd675b6-m524m"]
Sep 18 18:49:02.301515 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.301485    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fbd675b6-m524m"
Sep 18 18:49:02.387020 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.386994    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79502e36-ec13-40e1-8510-82fdef1c0573-webhook-cert\") pod \"maas-controller-75dc6cbdf-fx58b\" (UID: \"79502e36-ec13-40e1-8510-82fdef1c0573\") " pod="opendatahub/maas-controller-75dc6cbdf-fx58b"
Sep 18 18:49:02.387168 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.387030    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79502e36-ec13-40e1-8510-82fdef1c0573-metrics-certs\") pod \"maas-controller-75dc6cbdf-fx58b\" (UID: \"79502e36-ec13-40e1-8510-82fdef1c0573\") " pod="opendatahub/maas-controller-75dc6cbdf-fx58b"
Sep 18 18:49:02.387168 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.387051    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5589a623-805c-4e87-bd9c-8395aa27c58e-webhook-cert\") pod \"maas-controller-64fbd675b6-m524m\" (UID: \"5589a623-805c-4e87-bd9c-8395aa27c58e\") " pod="opendatahub/maas-controller-64fbd675b6-m524m"
Sep 18 18:49:02.387168 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.387114    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5jpr\" (UniqueName: \"kubernetes.io/projected/5589a623-805c-4e87-bd9c-8395aa27c58e-kube-api-access-h5jpr\") pod \"maas-controller-64fbd675b6-m524m\" (UID: \"5589a623-805c-4e87-bd9c-8395aa27c58e\") " pod="opendatahub/maas-controller-64fbd675b6-m524m"
Sep 18 18:49:02.387311 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:49:02.387164    2558 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 18 18:49:02.387311 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.387215    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcvp5\" (UniqueName: \"kubernetes.io/projected/79502e36-ec13-40e1-8510-82fdef1c0573-kube-api-access-jcvp5\") pod \"maas-controller-75dc6cbdf-fx58b\" (UID: \"79502e36-ec13-40e1-8510-82fdef1c0573\") " pod="opendatahub/maas-controller-75dc6cbdf-fx58b"
Sep 18 18:49:02.387311 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:49:02.387238    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79502e36-ec13-40e1-8510-82fdef1c0573-webhook-cert podName:79502e36-ec13-40e1-8510-82fdef1c0573 nodeName:}" failed. No retries permitted until 2026-09-18 18:49:02.88721898 +0000 UTC m=+579.085703164 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/79502e36-ec13-40e1-8510-82fdef1c0573-webhook-cert") pod "maas-controller-75dc6cbdf-fx58b" (UID: "79502e36-ec13-40e1-8510-82fdef1c0573") : secret "maas-controller-webhook-cert" not found
Sep 18 18:49:02.387311 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.387281    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5589a623-805c-4e87-bd9c-8395aa27c58e-metrics-certs\") pod \"maas-controller-64fbd675b6-m524m\" (UID: \"5589a623-805c-4e87-bd9c-8395aa27c58e\") " pod="opendatahub/maas-controller-64fbd675b6-m524m"
Sep 18 18:49:02.389525 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.389506    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79502e36-ec13-40e1-8510-82fdef1c0573-metrics-certs\") pod \"maas-controller-75dc6cbdf-fx58b\" (UID: \"79502e36-ec13-40e1-8510-82fdef1c0573\") " pod="opendatahub/maas-controller-75dc6cbdf-fx58b"
Sep 18 18:49:02.395739 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.395720    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcvp5\" (UniqueName: \"kubernetes.io/projected/79502e36-ec13-40e1-8510-82fdef1c0573-kube-api-access-jcvp5\") pod \"maas-controller-75dc6cbdf-fx58b\" (UID: \"79502e36-ec13-40e1-8510-82fdef1c0573\") " pod="opendatahub/maas-controller-75dc6cbdf-fx58b"
Sep 18 18:49:02.443486 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.443455    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:49:02.488248 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.488223    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5jpr\" (UniqueName: \"kubernetes.io/projected/5589a623-805c-4e87-bd9c-8395aa27c58e-kube-api-access-h5jpr\") pod \"maas-controller-64fbd675b6-m524m\" (UID: \"5589a623-805c-4e87-bd9c-8395aa27c58e\") " pod="opendatahub/maas-controller-64fbd675b6-m524m"
Sep 18 18:49:02.488405 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.488383    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5589a623-805c-4e87-bd9c-8395aa27c58e-metrics-certs\") pod \"maas-controller-64fbd675b6-m524m\" (UID: \"5589a623-805c-4e87-bd9c-8395aa27c58e\") " pod="opendatahub/maas-controller-64fbd675b6-m524m"
Sep 18 18:49:02.488527 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.488497    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5589a623-805c-4e87-bd9c-8395aa27c58e-webhook-cert\") pod \"maas-controller-64fbd675b6-m524m\" (UID: \"5589a623-805c-4e87-bd9c-8395aa27c58e\") " pod="opendatahub/maas-controller-64fbd675b6-m524m"
Sep 18 18:49:02.490660 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.490635    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5589a623-805c-4e87-bd9c-8395aa27c58e-metrics-certs\") pod \"maas-controller-64fbd675b6-m524m\" (UID: \"5589a623-805c-4e87-bd9c-8395aa27c58e\") " pod="opendatahub/maas-controller-64fbd675b6-m524m"
Sep 18 18:49:02.490803 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.490784    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5589a623-805c-4e87-bd9c-8395aa27c58e-webhook-cert\") pod \"maas-controller-64fbd675b6-m524m\" (UID: \"5589a623-805c-4e87-bd9c-8395aa27c58e\") " pod="opendatahub/maas-controller-64fbd675b6-m524m"
Sep 18 18:49:02.496212 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.496193    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5jpr\" (UniqueName: \"kubernetes.io/projected/5589a623-805c-4e87-bd9c-8395aa27c58e-kube-api-access-h5jpr\") pod \"maas-controller-64fbd675b6-m524m\" (UID: \"5589a623-805c-4e87-bd9c-8395aa27c58e\") " pod="opendatahub/maas-controller-64fbd675b6-m524m"
Sep 18 18:49:02.612397 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.612322    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fbd675b6-m524m"
Sep 18 18:49:02.892229 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.892150    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79502e36-ec13-40e1-8510-82fdef1c0573-webhook-cert\") pod \"maas-controller-75dc6cbdf-fx58b\" (UID: \"79502e36-ec13-40e1-8510-82fdef1c0573\") " pod="opendatahub/maas-controller-75dc6cbdf-fx58b"
Sep 18 18:49:02.894511 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.894488    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79502e36-ec13-40e1-8510-82fdef1c0573-webhook-cert\") pod \"maas-controller-75dc6cbdf-fx58b\" (UID: \"79502e36-ec13-40e1-8510-82fdef1c0573\") " pod="opendatahub/maas-controller-75dc6cbdf-fx58b"
Sep 18 18:49:02.939158 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:02.939138    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64fbd675b6-m524m"]
Sep 18 18:49:02.941031 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:49:02.941004    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5589a623_805c_4e87_bd9c_8395aa27c58e.slice/crio-3aca788a94e6a70a69bd2942a57a275d0c4be79f2dd5fbb82b535436b8c310bc WatchSource:0}: Error finding container 3aca788a94e6a70a69bd2942a57a275d0c4be79f2dd5fbb82b535436b8c310bc: Status 404 returned error can't find the container with id 3aca788a94e6a70a69bd2942a57a275d0c4be79f2dd5fbb82b535436b8c310bc
Sep 18 18:49:03.125325 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:03.125293    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-fx58b"
Sep 18 18:49:03.424000 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:03.423975    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-fx58b"]
Sep 18 18:49:03.426024 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:49:03.425995    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79502e36_ec13_40e1_8510_82fdef1c0573.slice/crio-0fb453ed149b5ebc34126ddd78009b1fe00c4f0743faee9174589b6709473fc4 WatchSource:0}: Error finding container 0fb453ed149b5ebc34126ddd78009b1fe00c4f0743faee9174589b6709473fc4: Status 404 returned error can't find the container with id 0fb453ed149b5ebc34126ddd78009b1fe00c4f0743faee9174589b6709473fc4
Sep 18 18:49:03.543465 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:03.543426    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fbd675b6-m524m" event={"ID":"5589a623-805c-4e87-bd9c-8395aa27c58e","Type":"ContainerStarted","Data":"3aca788a94e6a70a69bd2942a57a275d0c4be79f2dd5fbb82b535436b8c310bc"}
Sep 18 18:49:03.545102 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:03.545074    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-fx58b" event={"ID":"79502e36-ec13-40e1-8510-82fdef1c0573","Type":"ContainerStarted","Data":"0fb453ed149b5ebc34126ddd78009b1fe00c4f0743faee9174589b6709473fc4"}
Sep 18 18:49:05.702698 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:05.702659    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:49:05.816265 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:05.816186    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:49:06.562872 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:06.562832    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-fx58b" event={"ID":"79502e36-ec13-40e1-8510-82fdef1c0573","Type":"ContainerStarted","Data":"93e6f1a8d9315c6119378e1ef0cba6882867f9244a5ae675fb9641d23970a440"}
Sep 18 18:49:06.563074 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:06.562916    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75dc6cbdf-fx58b"
Sep 18 18:49:06.564472 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:06.564445    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fbd675b6-m524m" event={"ID":"5589a623-805c-4e87-bd9c-8395aa27c58e","Type":"ContainerStarted","Data":"cce37bd1e37dda8a7e9a8d53f0a724adf519b4c3a105256a65129e09184d2c8a"}
Sep 18 18:49:06.564604 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:06.564545    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64fbd675b6-m524m"
Sep 18 18:49:06.583665 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:06.583616    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75dc6cbdf-fx58b" podStartSLOduration=2.472738928 podStartE2EDuration="4.583602851s" totalImagesPullingTime="2.110863923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:49:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:49:06.582011747 +0000 UTC m=+582.780495969" watchObservedRunningTime="2026-09-18 18:49:06.583602851 +0000 UTC m=+582.782087056"
Sep 18 18:49:06.600000 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:06.599960    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64fbd675b6-m524m" podStartSLOduration=2.013933124 podStartE2EDuration="4.599949739s" totalImagesPullingTime="2.586016615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:49:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:49:06.598934366 +0000 UTC m=+582.797418596" watchObservedRunningTime="2026-09-18 18:49:06.599949739 +0000 UTC m=+582.798433977"
Sep 18 18:49:17.571953 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:17.571924    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64fbd675b6-m524m"
Sep 18 18:49:17.572329 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:17.572003    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75dc6cbdf-fx58b"
Sep 18 18:49:17.637812 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:17.637778    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-fx58b"]
Sep 18 18:49:17.638065 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:17.638015    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75dc6cbdf-fx58b" podUID="79502e36-ec13-40e1-8510-82fdef1c0573" containerName="manager" containerID="cri-o://93e6f1a8d9315c6119378e1ef0cba6882867f9244a5ae675fb9641d23970a440" gracePeriod=30
Sep 18 18:49:17.875732 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:17.875710    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-fx58b"
Sep 18 18:49:18.020538 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:18.020509    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79502e36-ec13-40e1-8510-82fdef1c0573-metrics-certs\") pod \"79502e36-ec13-40e1-8510-82fdef1c0573\" (UID: \"79502e36-ec13-40e1-8510-82fdef1c0573\") "
Sep 18 18:49:18.020720 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:18.020549    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcvp5\" (UniqueName: \"kubernetes.io/projected/79502e36-ec13-40e1-8510-82fdef1c0573-kube-api-access-jcvp5\") pod \"79502e36-ec13-40e1-8510-82fdef1c0573\" (UID: \"79502e36-ec13-40e1-8510-82fdef1c0573\") "
Sep 18 18:49:18.020720 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:18.020668    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79502e36-ec13-40e1-8510-82fdef1c0573-webhook-cert\") pod \"79502e36-ec13-40e1-8510-82fdef1c0573\" (UID: \"79502e36-ec13-40e1-8510-82fdef1c0573\") "
Sep 18 18:49:18.022722 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:18.022691    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79502e36-ec13-40e1-8510-82fdef1c0573-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "79502e36-ec13-40e1-8510-82fdef1c0573" (UID: "79502e36-ec13-40e1-8510-82fdef1c0573"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:49:18.022839 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:18.022753    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79502e36-ec13-40e1-8510-82fdef1c0573-kube-api-access-jcvp5" (OuterVolumeSpecName: "kube-api-access-jcvp5") pod "79502e36-ec13-40e1-8510-82fdef1c0573" (UID: "79502e36-ec13-40e1-8510-82fdef1c0573"). InnerVolumeSpecName "kube-api-access-jcvp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:49:18.022886 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:18.022843    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79502e36-ec13-40e1-8510-82fdef1c0573-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "79502e36-ec13-40e1-8510-82fdef1c0573" (UID: "79502e36-ec13-40e1-8510-82fdef1c0573"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:49:18.122362 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:18.122283    2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79502e36-ec13-40e1-8510-82fdef1c0573-webhook-cert\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:49:18.122362 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:18.122310    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79502e36-ec13-40e1-8510-82fdef1c0573-metrics-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:49:18.122362 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:18.122323    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcvp5\" (UniqueName: \"kubernetes.io/projected/79502e36-ec13-40e1-8510-82fdef1c0573-kube-api-access-jcvp5\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:49:18.620931 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:18.620891    2558 generic.go:356] "Generic (PLEG): container finished" podID="79502e36-ec13-40e1-8510-82fdef1c0573" containerID="93e6f1a8d9315c6119378e1ef0cba6882867f9244a5ae675fb9641d23970a440" exitCode=0
Sep 18 18:49:18.621364 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:18.620939    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-fx58b"
Sep 18 18:49:18.621364 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:18.620943    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-fx58b" event={"ID":"79502e36-ec13-40e1-8510-82fdef1c0573","Type":"ContainerDied","Data":"93e6f1a8d9315c6119378e1ef0cba6882867f9244a5ae675fb9641d23970a440"}
Sep 18 18:49:18.621364 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:18.620994    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-fx58b" event={"ID":"79502e36-ec13-40e1-8510-82fdef1c0573","Type":"ContainerDied","Data":"0fb453ed149b5ebc34126ddd78009b1fe00c4f0743faee9174589b6709473fc4"}
Sep 18 18:49:18.621364 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:18.621025    2558 scope.go:117] "RemoveContainer" containerID="93e6f1a8d9315c6119378e1ef0cba6882867f9244a5ae675fb9641d23970a440"
Sep 18 18:49:18.630206 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:18.630188    2558 scope.go:117] "RemoveContainer" containerID="93e6f1a8d9315c6119378e1ef0cba6882867f9244a5ae675fb9641d23970a440"
Sep 18 18:49:18.630478 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:49:18.630460    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93e6f1a8d9315c6119378e1ef0cba6882867f9244a5ae675fb9641d23970a440\": container with ID starting with 93e6f1a8d9315c6119378e1ef0cba6882867f9244a5ae675fb9641d23970a440 not found: ID does not exist" containerID="93e6f1a8d9315c6119378e1ef0cba6882867f9244a5ae675fb9641d23970a440"
Sep 18 18:49:18.630517 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:18.630487    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93e6f1a8d9315c6119378e1ef0cba6882867f9244a5ae675fb9641d23970a440"} err="failed to get container status \"93e6f1a8d9315c6119378e1ef0cba6882867f9244a5ae675fb9641d23970a440\": rpc error: code = NotFound desc = could not find container \"93e6f1a8d9315c6119378e1ef0cba6882867f9244a5ae675fb9641d23970a440\": container with ID starting with 93e6f1a8d9315c6119378e1ef0cba6882867f9244a5ae675fb9641d23970a440 not found: ID does not exist"
Sep 18 18:49:18.642717 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:18.642696    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-fx58b"]
Sep 18 18:49:18.646995 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:18.646975    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-fx58b"]
Sep 18 18:49:19.844342 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:19.844312    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79502e36-ec13-40e1-8510-82fdef1c0573" path="/var/lib/kubelet/pods/79502e36-ec13-40e1-8510-82fdef1c0573/volumes"
Sep 18 18:49:25.744343 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:25.744316    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bwsg_636f0979-fe2e-49fd-86d6-31bd7961e736/ovn-acl-logging/0.log"
Sep 18 18:49:25.773662 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:25.773639    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bwsg_636f0979-fe2e-49fd-86d6-31bd7961e736/ovn-acl-logging/0.log"
Sep 18 18:49:37.262128 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:37.262044    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:49:38.795607 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:38.795568    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"]
Sep 18 18:49:38.798112 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:38.796122    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79502e36-ec13-40e1-8510-82fdef1c0573" containerName="manager"
Sep 18 18:49:38.798112 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:38.796138    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="79502e36-ec13-40e1-8510-82fdef1c0573" containerName="manager"
Sep 18 18:49:38.798112 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:38.796206    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="79502e36-ec13-40e1-8510-82fdef1c0573" containerName="manager"
Sep 18 18:49:38.799385 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:38.799366    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 18:49:38.801892 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:38.801862    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 18:49:38.802003 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:38.801885    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-htbnh\""
Sep 18 18:49:38.802003 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:38.801966    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 18 18:49:38.809601 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:38.809578    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"]
Sep 18 18:49:38.870711 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:38.870667    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:49:38.899719 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:38.899692    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d69ce8d-b45d-48d9-a741-03dd8d21a684-metrics-certs\") pod \"maas-api-59f6566bbc-zx4j5\" (UID: \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 18:49:38.899865 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:38.899740    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d69ce8d-b45d-48d9-a741-03dd8d21a684-maas-api-tls\") pod \"maas-api-59f6566bbc-zx4j5\" (UID: \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 18:49:38.899865 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:38.899767    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n76mp\" (UniqueName: \"kubernetes.io/projected/2d69ce8d-b45d-48d9-a741-03dd8d21a684-kube-api-access-n76mp\") pod \"maas-api-59f6566bbc-zx4j5\" (UID: \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 18:49:38.899865 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:38.899805    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d69ce8d-b45d-48d9-a741-03dd8d21a684-openshift-service-ca\") pod \"maas-api-59f6566bbc-zx4j5\" (UID: \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 18:49:39.000248 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.000221    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d69ce8d-b45d-48d9-a741-03dd8d21a684-metrics-certs\") pod \"maas-api-59f6566bbc-zx4j5\" (UID: \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 18:49:39.000430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.000274    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d69ce8d-b45d-48d9-a741-03dd8d21a684-maas-api-tls\") pod \"maas-api-59f6566bbc-zx4j5\" (UID: \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 18:49:39.000430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.000298    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n76mp\" (UniqueName: \"kubernetes.io/projected/2d69ce8d-b45d-48d9-a741-03dd8d21a684-kube-api-access-n76mp\") pod \"maas-api-59f6566bbc-zx4j5\" (UID: \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 18:49:39.000430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.000325    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d69ce8d-b45d-48d9-a741-03dd8d21a684-openshift-service-ca\") pod \"maas-api-59f6566bbc-zx4j5\" (UID: \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 18:49:39.000430 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:49:39.000398    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 18 18:49:39.000612 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:49:39.000474    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d69ce8d-b45d-48d9-a741-03dd8d21a684-metrics-certs podName:2d69ce8d-b45d-48d9-a741-03dd8d21a684 nodeName:}" failed. No retries permitted until 2026-09-18 18:49:39.500450999 +0000 UTC m=+615.698935209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2d69ce8d-b45d-48d9-a741-03dd8d21a684-metrics-certs") pod "maas-api-59f6566bbc-zx4j5" (UID: "2d69ce8d-b45d-48d9-a741-03dd8d21a684") : secret "maas-api-metrics-tls" not found
Sep 18 18:49:39.001119 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.001092    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d69ce8d-b45d-48d9-a741-03dd8d21a684-openshift-service-ca\") pod \"maas-api-59f6566bbc-zx4j5\" (UID: \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 18:49:39.002722 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.002697    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d69ce8d-b45d-48d9-a741-03dd8d21a684-maas-api-tls\") pod \"maas-api-59f6566bbc-zx4j5\" (UID: \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 18:49:39.011456 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.011438    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n76mp\" (UniqueName: \"kubernetes.io/projected/2d69ce8d-b45d-48d9-a741-03dd8d21a684-kube-api-access-n76mp\") pod \"maas-api-59f6566bbc-zx4j5\" (UID: \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 18:49:39.050629 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.050556    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:49:39.151716 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.151665    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:49:39.244907 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.244877    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn"]
Sep 18 18:49:39.245276 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.245250    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:49:39.248420 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.248399    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn"
Sep 18 18:49:39.251080 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.251060    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-qzgnc\""
Sep 18 18:49:39.251080 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.251073    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 18 18:49:39.259547 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.259523    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn"]
Sep 18 18:49:39.302798 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.302716    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c20c633b-cb18-4a28-a1ef-790869c5893c-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-hz7jn\" (UID: \"c20c633b-cb18-4a28-a1ef-790869c5893c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn"
Sep 18 18:49:39.302798 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.302754    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snbg4\" (UniqueName: \"kubernetes.io/projected/c20c633b-cb18-4a28-a1ef-790869c5893c-kube-api-access-snbg4\") pod \"payload-pre-processing-59fbc47bf8-hz7jn\" (UID: \"c20c633b-cb18-4a28-a1ef-790869c5893c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn"
Sep 18 18:49:39.404395 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.404356    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c20c633b-cb18-4a28-a1ef-790869c5893c-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-hz7jn\" (UID: \"c20c633b-cb18-4a28-a1ef-790869c5893c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn"
Sep 18 18:49:39.404550 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.404412    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snbg4\" (UniqueName: \"kubernetes.io/projected/c20c633b-cb18-4a28-a1ef-790869c5893c-kube-api-access-snbg4\") pod \"payload-pre-processing-59fbc47bf8-hz7jn\" (UID: \"c20c633b-cb18-4a28-a1ef-790869c5893c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn"
Sep 18 18:49:39.405081 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.405061    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c20c633b-cb18-4a28-a1ef-790869c5893c-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-hz7jn\" (UID: \"c20c633b-cb18-4a28-a1ef-790869c5893c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn"
Sep 18 18:49:39.413537 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.413510    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snbg4\" (UniqueName: \"kubernetes.io/projected/c20c633b-cb18-4a28-a1ef-790869c5893c-kube-api-access-snbg4\") pod \"payload-pre-processing-59fbc47bf8-hz7jn\" (UID: \"c20c633b-cb18-4a28-a1ef-790869c5893c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn"
Sep 18 18:49:39.505653 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.505613    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d69ce8d-b45d-48d9-a741-03dd8d21a684-metrics-certs\") pod \"maas-api-59f6566bbc-zx4j5\" (UID: \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 18:49:39.508047 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.508017    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d69ce8d-b45d-48d9-a741-03dd8d21a684-metrics-certs\") pod \"maas-api-59f6566bbc-zx4j5\" (UID: \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 18:49:39.559570 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.559509    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn"
Sep 18 18:49:39.710235 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.710204    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 18:49:39.832843 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.832805    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn"]
Sep 18 18:49:39.834798 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:49:39.834769    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc20c633b_cb18_4a28_a1ef_790869c5893c.slice/crio-f1a17f0807a9e8bfd22e2615464086a19301ce8769f05f35591a65c1983a26aa WatchSource:0}: Error finding container f1a17f0807a9e8bfd22e2615464086a19301ce8769f05f35591a65c1983a26aa: Status 404 returned error can't find the container with id f1a17f0807a9e8bfd22e2615464086a19301ce8769f05f35591a65c1983a26aa
Sep 18 18:49:39.982623 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:39.982585    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"]
Sep 18 18:49:39.985180 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:49:39.985148    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d69ce8d_b45d_48d9_a741_03dd8d21a684.slice/crio-4992013b8423bd1a1fed1b73c943a3ed5e6fd9592078ed1d63a6958a50d3ecc3 WatchSource:0}: Error finding container 4992013b8423bd1a1fed1b73c943a3ed5e6fd9592078ed1d63a6958a50d3ecc3: Status 404 returned error can't find the container with id 4992013b8423bd1a1fed1b73c943a3ed5e6fd9592078ed1d63a6958a50d3ecc3
Sep 18 18:49:40.744220 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:40.744182    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn" event={"ID":"c20c633b-cb18-4a28-a1ef-790869c5893c","Type":"ContainerStarted","Data":"f1a17f0807a9e8bfd22e2615464086a19301ce8769f05f35591a65c1983a26aa"}
Sep 18 18:49:40.745658 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:40.745635    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5" event={"ID":"2d69ce8d-b45d-48d9-a741-03dd8d21a684","Type":"ContainerStarted","Data":"4992013b8423bd1a1fed1b73c943a3ed5e6fd9592078ed1d63a6958a50d3ecc3"}
Sep 18 18:49:41.758646 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:41.758590    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn" event={"ID":"c20c633b-cb18-4a28-a1ef-790869c5893c","Type":"ContainerStarted","Data":"da50eba50e8926d9ee79c86e17cc10cd98a411d30013d914bade42194b63c0ab"}
Sep 18 18:49:41.759094 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:41.758906    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn"
Sep 18 18:49:41.760236 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:41.760212    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5" event={"ID":"2d69ce8d-b45d-48d9-a741-03dd8d21a684","Type":"ContainerStarted","Data":"ba7a1c60c8a3b804ac744522b787ea85879c42810a48c92b77b8fb43c1b3cc60"}
Sep 18 18:49:41.760365 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:41.760350    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 18:49:41.777856 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:41.777792    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn" podStartSLOduration=0.983397276 podStartE2EDuration="2.777778016s" totalImagesPullingTime="1.79438074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:49:41.776840742 +0000 UTC m=+617.975324977" watchObservedRunningTime="2026-09-18 18:49:41.777778016 +0000 UTC m=+617.976262219"
Sep 18 18:49:41.794065 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:41.794023    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5" podStartSLOduration=2.146598571 podStartE2EDuration="3.794009323s" totalImagesPullingTime="1.647410752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:49:41.791928986 +0000 UTC m=+617.990413215" watchObservedRunningTime="2026-09-18 18:49:41.794009323 +0000 UTC m=+617.992493527"
Sep 18 18:49:47.770771 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:47.770740    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 18:49:49.288004 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:49.287968    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:49:49.387379 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:49.387342    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:49:52.767006 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:52.766976    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn"
Sep 18 18:49:58.053129 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.053099    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"]
Sep 18 18:49:58.056875 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.056855    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.060248 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.060231    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 18 18:49:58.061099 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.061071    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 18 18:49:58.061208 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.061130    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 18 18:49:58.061443 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.061422    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-c6hs5\""
Sep 18 18:49:58.068073 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.068053    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"]
Sep 18 18:49:58.079758 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.079731    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:49:58.178010 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.177972    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wm5n\" (UniqueName: \"kubernetes.io/projected/cd12e6f2-8f6e-4232-a1af-9480f385113a-kube-api-access-2wm5n\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5\" (UID: \"cd12e6f2-8f6e-4232-a1af-9480f385113a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.178165 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.178018    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd12e6f2-8f6e-4232-a1af-9480f385113a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5\" (UID: \"cd12e6f2-8f6e-4232-a1af-9480f385113a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.178165 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.178063    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd12e6f2-8f6e-4232-a1af-9480f385113a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5\" (UID: \"cd12e6f2-8f6e-4232-a1af-9480f385113a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.178165 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.178081    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd12e6f2-8f6e-4232-a1af-9480f385113a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5\" (UID: \"cd12e6f2-8f6e-4232-a1af-9480f385113a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.178165 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.178117    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd12e6f2-8f6e-4232-a1af-9480f385113a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5\" (UID: \"cd12e6f2-8f6e-4232-a1af-9480f385113a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.178165 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.178138    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd12e6f2-8f6e-4232-a1af-9480f385113a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5\" (UID: \"cd12e6f2-8f6e-4232-a1af-9480f385113a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.279022 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.278988    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wm5n\" (UniqueName: \"kubernetes.io/projected/cd12e6f2-8f6e-4232-a1af-9480f385113a-kube-api-access-2wm5n\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5\" (UID: \"cd12e6f2-8f6e-4232-a1af-9480f385113a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.279209 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.279032    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd12e6f2-8f6e-4232-a1af-9480f385113a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5\" (UID: \"cd12e6f2-8f6e-4232-a1af-9480f385113a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.279209 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.279051    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd12e6f2-8f6e-4232-a1af-9480f385113a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5\" (UID: \"cd12e6f2-8f6e-4232-a1af-9480f385113a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.279209 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.279067    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd12e6f2-8f6e-4232-a1af-9480f385113a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5\" (UID: \"cd12e6f2-8f6e-4232-a1af-9480f385113a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.279209 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.279098    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd12e6f2-8f6e-4232-a1af-9480f385113a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5\" (UID: \"cd12e6f2-8f6e-4232-a1af-9480f385113a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.279209 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.279114    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd12e6f2-8f6e-4232-a1af-9480f385113a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5\" (UID: \"cd12e6f2-8f6e-4232-a1af-9480f385113a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.279480 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.279456    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd12e6f2-8f6e-4232-a1af-9480f385113a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5\" (UID: \"cd12e6f2-8f6e-4232-a1af-9480f385113a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.279480 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.279453    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd12e6f2-8f6e-4232-a1af-9480f385113a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5\" (UID: \"cd12e6f2-8f6e-4232-a1af-9480f385113a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.279590 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.279529    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd12e6f2-8f6e-4232-a1af-9480f385113a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5\" (UID: \"cd12e6f2-8f6e-4232-a1af-9480f385113a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.281402 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.281360    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd12e6f2-8f6e-4232-a1af-9480f385113a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5\" (UID: \"cd12e6f2-8f6e-4232-a1af-9480f385113a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.281646 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.281625    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd12e6f2-8f6e-4232-a1af-9480f385113a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5\" (UID: \"cd12e6f2-8f6e-4232-a1af-9480f385113a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.286794 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.286768    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wm5n\" (UniqueName: \"kubernetes.io/projected/cd12e6f2-8f6e-4232-a1af-9480f385113a-kube-api-access-2wm5n\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5\" (UID: \"cd12e6f2-8f6e-4232-a1af-9480f385113a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.368765 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.368688    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:49:58.639159 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.639133    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"]
Sep 18 18:49:58.641038 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:49:58.641012    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd12e6f2_8f6e_4232_a1af_9480f385113a.slice/crio-da56d1c6ed0b2b7498a28560d848a7f5bac46aeb3496e0d7c33f27e31bf58fc2 WatchSource:0}: Error finding container da56d1c6ed0b2b7498a28560d848a7f5bac46aeb3496e0d7c33f27e31bf58fc2: Status 404 returned error can't find the container with id da56d1c6ed0b2b7498a28560d848a7f5bac46aeb3496e0d7c33f27e31bf58fc2
Sep 18 18:49:58.642436 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.642415    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:49:58.842174 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:58.842137    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5" event={"ID":"cd12e6f2-8f6e-4232-a1af-9480f385113a","Type":"ContainerStarted","Data":"da56d1c6ed0b2b7498a28560d848a7f5bac46aeb3496e0d7c33f27e31bf58fc2"}
Sep 18 18:49:59.848547 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:59.848508    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5" event={"ID":"cd12e6f2-8f6e-4232-a1af-9480f385113a","Type":"ContainerStarted","Data":"67865058e67502fe45032208c4ae9b279a7fb7aa2a1aa7b8af67bacdfe31ce5e"}
Sep 18 18:49:59.865883 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:49:59.865814    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5" podStartSLOduration=0.833697806 podStartE2EDuration="1.865797722s" totalImagesPullingTime="1.032099916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:49:59.865149099 +0000 UTC m=+636.063633317" watchObservedRunningTime="2026-09-18 18:49:59.865797722 +0000 UTC m=+636.064281926"
Sep 18 18:50:08.369249 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:50:08.369217    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:50:08.369723 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:50:08.369279    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:50:08.381435 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:50:08.381408    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:50:08.902504 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:50:08.902471    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zhgt5"
Sep 18 18:50:11.476583 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:50:11.476545    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:50:26.881705 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:50:26.881671    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:50:34.139257 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:50:34.139215    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:50:36.652380 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:50:36.652347    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:50:54.268199 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:50:54.268162    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:50:54.369645 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:50:54.369607    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:50:54.574650 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:50:54.574573    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:51:08.851074 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.850983    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"]
Sep 18 18:51:08.853802 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.853781    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:08.856806 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.856782    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 18 18:51:08.865589 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.865563    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"]
Sep 18 18:51:08.885738 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.885715    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3dc5855-6c8b-4608-9a79-d5294d1711d7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kzt2k\" (UID: \"c3dc5855-6c8b-4608-9a79-d5294d1711d7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:08.885874 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.885771    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3dc5855-6c8b-4608-9a79-d5294d1711d7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kzt2k\" (UID: \"c3dc5855-6c8b-4608-9a79-d5294d1711d7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:08.885874 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.885790    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3dc5855-6c8b-4608-9a79-d5294d1711d7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kzt2k\" (UID: \"c3dc5855-6c8b-4608-9a79-d5294d1711d7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:08.885874 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.885813    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rrz4\" (UniqueName: \"kubernetes.io/projected/c3dc5855-6c8b-4608-9a79-d5294d1711d7-kube-api-access-6rrz4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kzt2k\" (UID: \"c3dc5855-6c8b-4608-9a79-d5294d1711d7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:08.886281 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.886219    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3dc5855-6c8b-4608-9a79-d5294d1711d7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kzt2k\" (UID: \"c3dc5855-6c8b-4608-9a79-d5294d1711d7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:08.886384 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.886283    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3dc5855-6c8b-4608-9a79-d5294d1711d7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kzt2k\" (UID: \"c3dc5855-6c8b-4608-9a79-d5294d1711d7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:08.886873 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.886846    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:51:08.987754 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.987724    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3dc5855-6c8b-4608-9a79-d5294d1711d7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kzt2k\" (UID: \"c3dc5855-6c8b-4608-9a79-d5294d1711d7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:08.987920 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.987756    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3dc5855-6c8b-4608-9a79-d5294d1711d7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kzt2k\" (UID: \"c3dc5855-6c8b-4608-9a79-d5294d1711d7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:08.987920 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.987780    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rrz4\" (UniqueName: \"kubernetes.io/projected/c3dc5855-6c8b-4608-9a79-d5294d1711d7-kube-api-access-6rrz4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kzt2k\" (UID: \"c3dc5855-6c8b-4608-9a79-d5294d1711d7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:08.987920 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.987882    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3dc5855-6c8b-4608-9a79-d5294d1711d7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kzt2k\" (UID: \"c3dc5855-6c8b-4608-9a79-d5294d1711d7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:08.988096 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.987919    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3dc5855-6c8b-4608-9a79-d5294d1711d7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kzt2k\" (UID: \"c3dc5855-6c8b-4608-9a79-d5294d1711d7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:08.988096 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.987984    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3dc5855-6c8b-4608-9a79-d5294d1711d7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kzt2k\" (UID: \"c3dc5855-6c8b-4608-9a79-d5294d1711d7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:08.988180 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.988133    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3dc5855-6c8b-4608-9a79-d5294d1711d7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kzt2k\" (UID: \"c3dc5855-6c8b-4608-9a79-d5294d1711d7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:08.988217 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.988193    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3dc5855-6c8b-4608-9a79-d5294d1711d7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kzt2k\" (UID: \"c3dc5855-6c8b-4608-9a79-d5294d1711d7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:08.988255 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.988215    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3dc5855-6c8b-4608-9a79-d5294d1711d7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kzt2k\" (UID: \"c3dc5855-6c8b-4608-9a79-d5294d1711d7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:08.990068 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.990052    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3dc5855-6c8b-4608-9a79-d5294d1711d7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kzt2k\" (UID: \"c3dc5855-6c8b-4608-9a79-d5294d1711d7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:08.990215 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.990200    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3dc5855-6c8b-4608-9a79-d5294d1711d7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kzt2k\" (UID: \"c3dc5855-6c8b-4608-9a79-d5294d1711d7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:08.995781 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:08.995760    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rrz4\" (UniqueName: \"kubernetes.io/projected/c3dc5855-6c8b-4608-9a79-d5294d1711d7-kube-api-access-6rrz4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kzt2k\" (UID: \"c3dc5855-6c8b-4608-9a79-d5294d1711d7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:09.166430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:09.166349    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:09.437238 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:09.437159    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"]
Sep 18 18:51:09.440647 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:51:09.440618    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3dc5855_6c8b_4608_9a79_d5294d1711d7.slice/crio-5b9767af1283f8eabbd440b095cd72466b112f1818ad180cafcac54cdc349b2c WatchSource:0}: Error finding container 5b9767af1283f8eabbd440b095cd72466b112f1818ad180cafcac54cdc349b2c: Status 404 returned error can't find the container with id 5b9767af1283f8eabbd440b095cd72466b112f1818ad180cafcac54cdc349b2c
Sep 18 18:51:10.198315 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:10.198276    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k" event={"ID":"c3dc5855-6c8b-4608-9a79-d5294d1711d7","Type":"ContainerStarted","Data":"eb57a98af58d57b39b54b2f5698904432d08f0b03553259c2ec0f7e362e1b64c"}
Sep 18 18:51:10.198688 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:10.198333    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k" event={"ID":"c3dc5855-6c8b-4608-9a79-d5294d1711d7","Type":"ContainerStarted","Data":"5b9767af1283f8eabbd440b095cd72466b112f1818ad180cafcac54cdc349b2c"}
Sep 18 18:51:10.227589 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:10.227544    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k" podStartSLOduration=2.059090721 podStartE2EDuration="2.227524626s" totalImagesPullingTime="168.433905ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:51:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:51:10.22651763 +0000 UTC m=+706.425001882" watchObservedRunningTime="2026-09-18 18:51:10.227524626 +0000 UTC m=+706.426008833"
Sep 18 18:51:19.167528 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:19.167495    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:19.167920 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:19.167557    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:19.179711 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:19.179690    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:19.251979 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:19.251953    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kzt2k"
Sep 18 18:51:19.497860 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:51:19.497801    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:52:00.477983 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:00.477944    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:52:00.538684 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:00.538636    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:52:22.072092 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:22.072058    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dc55d7c-wdjw8"]
Sep 18 18:52:22.080998 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:22.080965    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dc55d7c-wdjw8"
Sep 18 18:52:22.082655 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:22.082628    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dc55d7c-wdjw8"]
Sep 18 18:52:22.084581 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:22.084562    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6xsp7\""
Sep 18 18:52:22.084687 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:22.084564    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 18 18:52:22.238233 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:22.238202    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/50d75267-8a6d-4bd5-9070-1df3879af1ab-tls-cert\") pod \"authorino-6d4dc55d7c-wdjw8\" (UID: \"50d75267-8a6d-4bd5-9070-1df3879af1ab\") " pod="kuadrant-system/authorino-6d4dc55d7c-wdjw8"
Sep 18 18:52:22.238406 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:22.238258    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk9pj\" (UniqueName: \"kubernetes.io/projected/50d75267-8a6d-4bd5-9070-1df3879af1ab-kube-api-access-hk9pj\") pod \"authorino-6d4dc55d7c-wdjw8\" (UID: \"50d75267-8a6d-4bd5-9070-1df3879af1ab\") " pod="kuadrant-system/authorino-6d4dc55d7c-wdjw8"
Sep 18 18:52:22.339078 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:22.339010    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/50d75267-8a6d-4bd5-9070-1df3879af1ab-tls-cert\") pod \"authorino-6d4dc55d7c-wdjw8\" (UID: \"50d75267-8a6d-4bd5-9070-1df3879af1ab\") " pod="kuadrant-system/authorino-6d4dc55d7c-wdjw8"
Sep 18 18:52:22.339078 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:22.339067    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk9pj\" (UniqueName: \"kubernetes.io/projected/50d75267-8a6d-4bd5-9070-1df3879af1ab-kube-api-access-hk9pj\") pod \"authorino-6d4dc55d7c-wdjw8\" (UID: \"50d75267-8a6d-4bd5-9070-1df3879af1ab\") " pod="kuadrant-system/authorino-6d4dc55d7c-wdjw8"
Sep 18 18:52:22.341367 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:22.341338    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/50d75267-8a6d-4bd5-9070-1df3879af1ab-tls-cert\") pod \"authorino-6d4dc55d7c-wdjw8\" (UID: \"50d75267-8a6d-4bd5-9070-1df3879af1ab\") " pod="kuadrant-system/authorino-6d4dc55d7c-wdjw8"
Sep 18 18:52:22.346315 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:22.346294    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk9pj\" (UniqueName: \"kubernetes.io/projected/50d75267-8a6d-4bd5-9070-1df3879af1ab-kube-api-access-hk9pj\") pod \"authorino-6d4dc55d7c-wdjw8\" (UID: \"50d75267-8a6d-4bd5-9070-1df3879af1ab\") " pod="kuadrant-system/authorino-6d4dc55d7c-wdjw8"
Sep 18 18:52:22.391434 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:22.391408    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dc55d7c-wdjw8"
Sep 18 18:52:22.658207 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:22.658183    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dc55d7c-wdjw8"]
Sep 18 18:52:22.659981 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:52:22.659952    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50d75267_8a6d_4bd5_9070_1df3879af1ab.slice/crio-471357a899a9076b9ac767948c6c36bbefee99224c08beab49b24d62f26dfebc WatchSource:0}: Error finding container 471357a899a9076b9ac767948c6c36bbefee99224c08beab49b24d62f26dfebc: Status 404 returned error can't find the container with id 471357a899a9076b9ac767948c6c36bbefee99224c08beab49b24d62f26dfebc
Sep 18 18:52:23.563120 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:23.563090    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d4dc55d7c-wdjw8" event={"ID":"50d75267-8a6d-4bd5-9070-1df3879af1ab","Type":"ContainerStarted","Data":"eb57c0b4ecc4d944dc93c6b50c591d2772c2f2dba649cd7af3c8d16e13d74cf4"}
Sep 18 18:52:23.563486 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:23.563130    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d4dc55d7c-wdjw8" event={"ID":"50d75267-8a6d-4bd5-9070-1df3879af1ab","Type":"ContainerStarted","Data":"471357a899a9076b9ac767948c6c36bbefee99224c08beab49b24d62f26dfebc"}
Sep 18 18:52:23.578855 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:23.578791    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6d4dc55d7c-wdjw8" podStartSLOduration=0.744160671 podStartE2EDuration="1.57877381s" totalImagesPullingTime="834.613139ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:52:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:52:23.576906129 +0000 UTC m=+779.775390334" watchObservedRunningTime="2026-09-18 18:52:23.57877381 +0000 UTC m=+779.777258025"
Sep 18 18:52:23.940086 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:23.940004    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:52:24.115283 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.115252    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64fbd675b6-m524m"]
Sep 18 18:52:24.115511 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.115484    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-64fbd675b6-m524m" podUID="5589a623-805c-4e87-bd9c-8395aa27c58e" containerName="manager" containerID="cri-o://cce37bd1e37dda8a7e9a8d53f0a724adf519b4c3a105256a65129e09184d2c8a" gracePeriod=30
Sep 18 18:52:24.376651 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.376627    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fbd675b6-m524m"
Sep 18 18:52:24.424769 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.424727    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:52:24.465712 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.465676    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn"]
Sep 18 18:52:24.524706 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.524668    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:52:24.561605 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.561570    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5jpr\" (UniqueName: \"kubernetes.io/projected/5589a623-805c-4e87-bd9c-8395aa27c58e-kube-api-access-h5jpr\") pod \"5589a623-805c-4e87-bd9c-8395aa27c58e\" (UID: \"5589a623-805c-4e87-bd9c-8395aa27c58e\") "
Sep 18 18:52:24.561795 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.561726    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5589a623-805c-4e87-bd9c-8395aa27c58e-webhook-cert\") pod \"5589a623-805c-4e87-bd9c-8395aa27c58e\" (UID: \"5589a623-805c-4e87-bd9c-8395aa27c58e\") "
Sep 18 18:52:24.561795 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.561773    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5589a623-805c-4e87-bd9c-8395aa27c58e-metrics-certs\") pod \"5589a623-805c-4e87-bd9c-8395aa27c58e\" (UID: \"5589a623-805c-4e87-bd9c-8395aa27c58e\") "
Sep 18 18:52:24.563791 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.563764    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5589a623-805c-4e87-bd9c-8395aa27c58e-kube-api-access-h5jpr" (OuterVolumeSpecName: "kube-api-access-h5jpr") pod "5589a623-805c-4e87-bd9c-8395aa27c58e" (UID: "5589a623-805c-4e87-bd9c-8395aa27c58e"). InnerVolumeSpecName "kube-api-access-h5jpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:52:24.564138 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.563880    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5589a623-805c-4e87-bd9c-8395aa27c58e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5589a623-805c-4e87-bd9c-8395aa27c58e" (UID: "5589a623-805c-4e87-bd9c-8395aa27c58e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:52:24.564138 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.564009    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5589a623-805c-4e87-bd9c-8395aa27c58e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5589a623-805c-4e87-bd9c-8395aa27c58e" (UID: "5589a623-805c-4e87-bd9c-8395aa27c58e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:52:24.570749 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.570715    2558 generic.go:356] "Generic (PLEG): container finished" podID="5589a623-805c-4e87-bd9c-8395aa27c58e" containerID="cce37bd1e37dda8a7e9a8d53f0a724adf519b4c3a105256a65129e09184d2c8a" exitCode=0
Sep 18 18:52:24.570887 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.570775    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fbd675b6-m524m"
Sep 18 18:52:24.570887 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.570858    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fbd675b6-m524m" event={"ID":"5589a623-805c-4e87-bd9c-8395aa27c58e","Type":"ContainerDied","Data":"cce37bd1e37dda8a7e9a8d53f0a724adf519b4c3a105256a65129e09184d2c8a"}
Sep 18 18:52:24.571004 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.570908    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fbd675b6-m524m" event={"ID":"5589a623-805c-4e87-bd9c-8395aa27c58e","Type":"ContainerDied","Data":"3aca788a94e6a70a69bd2942a57a275d0c4be79f2dd5fbb82b535436b8c310bc"}
Sep 18 18:52:24.571004 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.570941    2558 scope.go:117] "RemoveContainer" containerID="cce37bd1e37dda8a7e9a8d53f0a724adf519b4c3a105256a65129e09184d2c8a"
Sep 18 18:52:24.571945 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.571199    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn" podUID="c20c633b-cb18-4a28-a1ef-790869c5893c" containerName="payload-pre-processing" containerID="cri-o://da50eba50e8926d9ee79c86e17cc10cd98a411d30013d914bade42194b63c0ab" gracePeriod=30
Sep 18 18:52:24.643676 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.643656    2558 scope.go:117] "RemoveContainer" containerID="cce37bd1e37dda8a7e9a8d53f0a724adf519b4c3a105256a65129e09184d2c8a"
Sep 18 18:52:24.644080 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:52:24.644059    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cce37bd1e37dda8a7e9a8d53f0a724adf519b4c3a105256a65129e09184d2c8a\": container with ID starting with cce37bd1e37dda8a7e9a8d53f0a724adf519b4c3a105256a65129e09184d2c8a not found: ID does not exist" containerID="cce37bd1e37dda8a7e9a8d53f0a724adf519b4c3a105256a65129e09184d2c8a"
Sep 18 18:52:24.644159 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.644094    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cce37bd1e37dda8a7e9a8d53f0a724adf519b4c3a105256a65129e09184d2c8a"} err="failed to get container status \"cce37bd1e37dda8a7e9a8d53f0a724adf519b4c3a105256a65129e09184d2c8a\": rpc error: code = NotFound desc = could not find container \"cce37bd1e37dda8a7e9a8d53f0a724adf519b4c3a105256a65129e09184d2c8a\": container with ID starting with cce37bd1e37dda8a7e9a8d53f0a724adf519b4c3a105256a65129e09184d2c8a not found: ID does not exist"
Sep 18 18:52:24.655528 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.655503    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64fbd675b6-m524m"]
Sep 18 18:52:24.659839 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.659802    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-64fbd675b6-m524m"]
Sep 18 18:52:24.663038 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.663012    2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5589a623-805c-4e87-bd9c-8395aa27c58e-webhook-cert\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:52:24.663038 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.663037    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5589a623-805c-4e87-bd9c-8395aa27c58e-metrics-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:52:24.663231 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.663047    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5jpr\" (UniqueName: \"kubernetes.io/projected/5589a623-805c-4e87-bd9c-8395aa27c58e-kube-api-access-h5jpr\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:52:24.829421 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.829399    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn"
Sep 18 18:52:24.965749 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.965716    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snbg4\" (UniqueName: \"kubernetes.io/projected/c20c633b-cb18-4a28-a1ef-790869c5893c-kube-api-access-snbg4\") pod \"c20c633b-cb18-4a28-a1ef-790869c5893c\" (UID: \"c20c633b-cb18-4a28-a1ef-790869c5893c\") "
Sep 18 18:52:24.965939 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.965813    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c20c633b-cb18-4a28-a1ef-790869c5893c-plugins-config-volume\") pod \"c20c633b-cb18-4a28-a1ef-790869c5893c\" (UID: \"c20c633b-cb18-4a28-a1ef-790869c5893c\") "
Sep 18 18:52:24.966273 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.966234    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c20c633b-cb18-4a28-a1ef-790869c5893c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c20c633b-cb18-4a28-a1ef-790869c5893c" (UID: "c20c633b-cb18-4a28-a1ef-790869c5893c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:52:24.967913 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:24.967888    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c20c633b-cb18-4a28-a1ef-790869c5893c-kube-api-access-snbg4" (OuterVolumeSpecName: "kube-api-access-snbg4") pod "c20c633b-cb18-4a28-a1ef-790869c5893c" (UID: "c20c633b-cb18-4a28-a1ef-790869c5893c"). InnerVolumeSpecName "kube-api-access-snbg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:52:25.067560 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.067477    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c20c633b-cb18-4a28-a1ef-790869c5893c-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:52:25.067560 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.067506    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snbg4\" (UniqueName: \"kubernetes.io/projected/c20c633b-cb18-4a28-a1ef-790869c5893c-kube-api-access-snbg4\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:52:25.133859 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.133808    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64fbd675b6-l9gft"]
Sep 18 18:52:25.134254 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.134234    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5589a623-805c-4e87-bd9c-8395aa27c58e" containerName="manager"
Sep 18 18:52:25.134254 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.134250    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5589a623-805c-4e87-bd9c-8395aa27c58e" containerName="manager"
Sep 18 18:52:25.134254 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.134258    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c20c633b-cb18-4a28-a1ef-790869c5893c" containerName="payload-pre-processing"
Sep 18 18:52:25.134439 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.134263    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c20c633b-cb18-4a28-a1ef-790869c5893c" containerName="payload-pre-processing"
Sep 18 18:52:25.134439 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.134333    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5589a623-805c-4e87-bd9c-8395aa27c58e" containerName="manager"
Sep 18 18:52:25.134439 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.134346    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c20c633b-cb18-4a28-a1ef-790869c5893c" containerName="payload-pre-processing"
Sep 18 18:52:25.137373 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.137351    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fbd675b6-l9gft"
Sep 18 18:52:25.139977 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.139952    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 18 18:52:25.140099 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.140004    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-x7kbw\""
Sep 18 18:52:25.140099 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.140006    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 18 18:52:25.140862 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.140803    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 18 18:52:25.146443 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.146411    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64fbd675b6-l9gft"]
Sep 18 18:52:25.269802 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.269746    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/939e465d-57fd-4592-a8e8-13abfabd5c1a-metrics-certs\") pod \"maas-controller-64fbd675b6-l9gft\" (UID: \"939e465d-57fd-4592-a8e8-13abfabd5c1a\") " pod="opendatahub/maas-controller-64fbd675b6-l9gft"
Sep 18 18:52:25.269802 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.269813    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/939e465d-57fd-4592-a8e8-13abfabd5c1a-webhook-cert\") pod \"maas-controller-64fbd675b6-l9gft\" (UID: \"939e465d-57fd-4592-a8e8-13abfabd5c1a\") " pod="opendatahub/maas-controller-64fbd675b6-l9gft"
Sep 18 18:52:25.270095 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.269878    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdjmh\" (UniqueName: \"kubernetes.io/projected/939e465d-57fd-4592-a8e8-13abfabd5c1a-kube-api-access-cdjmh\") pod \"maas-controller-64fbd675b6-l9gft\" (UID: \"939e465d-57fd-4592-a8e8-13abfabd5c1a\") " pod="opendatahub/maas-controller-64fbd675b6-l9gft"
Sep 18 18:52:25.370587 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.370487    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/939e465d-57fd-4592-a8e8-13abfabd5c1a-webhook-cert\") pod \"maas-controller-64fbd675b6-l9gft\" (UID: \"939e465d-57fd-4592-a8e8-13abfabd5c1a\") " pod="opendatahub/maas-controller-64fbd675b6-l9gft"
Sep 18 18:52:25.370587 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.370556    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdjmh\" (UniqueName: \"kubernetes.io/projected/939e465d-57fd-4592-a8e8-13abfabd5c1a-kube-api-access-cdjmh\") pod \"maas-controller-64fbd675b6-l9gft\" (UID: \"939e465d-57fd-4592-a8e8-13abfabd5c1a\") " pod="opendatahub/maas-controller-64fbd675b6-l9gft"
Sep 18 18:52:25.370813 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.370657    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/939e465d-57fd-4592-a8e8-13abfabd5c1a-metrics-certs\") pod \"maas-controller-64fbd675b6-l9gft\" (UID: \"939e465d-57fd-4592-a8e8-13abfabd5c1a\") " pod="opendatahub/maas-controller-64fbd675b6-l9gft"
Sep 18 18:52:25.375300 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.373744    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/939e465d-57fd-4592-a8e8-13abfabd5c1a-metrics-certs\") pod \"maas-controller-64fbd675b6-l9gft\" (UID: \"939e465d-57fd-4592-a8e8-13abfabd5c1a\") " pod="opendatahub/maas-controller-64fbd675b6-l9gft"
Sep 18 18:52:25.375300 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.374581    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/939e465d-57fd-4592-a8e8-13abfabd5c1a-webhook-cert\") pod \"maas-controller-64fbd675b6-l9gft\" (UID: \"939e465d-57fd-4592-a8e8-13abfabd5c1a\") " pod="opendatahub/maas-controller-64fbd675b6-l9gft"
Sep 18 18:52:25.384064 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.384033    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdjmh\" (UniqueName: \"kubernetes.io/projected/939e465d-57fd-4592-a8e8-13abfabd5c1a-kube-api-access-cdjmh\") pod \"maas-controller-64fbd675b6-l9gft\" (UID: \"939e465d-57fd-4592-a8e8-13abfabd5c1a\") " pod="opendatahub/maas-controller-64fbd675b6-l9gft"
Sep 18 18:52:25.448582 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.448552    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fbd675b6-l9gft"
Sep 18 18:52:25.578428 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.578398    2558 generic.go:356] "Generic (PLEG): container finished" podID="c20c633b-cb18-4a28-a1ef-790869c5893c" containerID="da50eba50e8926d9ee79c86e17cc10cd98a411d30013d914bade42194b63c0ab" exitCode=0
Sep 18 18:52:25.578876 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.578458    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn"
Sep 18 18:52:25.578876 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.578478    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn" event={"ID":"c20c633b-cb18-4a28-a1ef-790869c5893c","Type":"ContainerDied","Data":"da50eba50e8926d9ee79c86e17cc10cd98a411d30013d914bade42194b63c0ab"}
Sep 18 18:52:25.578876 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.578531    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn" event={"ID":"c20c633b-cb18-4a28-a1ef-790869c5893c","Type":"ContainerDied","Data":"f1a17f0807a9e8bfd22e2615464086a19301ce8769f05f35591a65c1983a26aa"}
Sep 18 18:52:25.578876 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.578561    2558 scope.go:117] "RemoveContainer" containerID="da50eba50e8926d9ee79c86e17cc10cd98a411d30013d914bade42194b63c0ab"
Sep 18 18:52:25.596319 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.596295    2558 scope.go:117] "RemoveContainer" containerID="da50eba50e8926d9ee79c86e17cc10cd98a411d30013d914bade42194b63c0ab"
Sep 18 18:52:25.596604 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:52:25.596579    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da50eba50e8926d9ee79c86e17cc10cd98a411d30013d914bade42194b63c0ab\": container with ID starting with da50eba50e8926d9ee79c86e17cc10cd98a411d30013d914bade42194b63c0ab not found: ID does not exist" containerID="da50eba50e8926d9ee79c86e17cc10cd98a411d30013d914bade42194b63c0ab"
Sep 18 18:52:25.596700 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.596609    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da50eba50e8926d9ee79c86e17cc10cd98a411d30013d914bade42194b63c0ab"} err="failed to get container status \"da50eba50e8926d9ee79c86e17cc10cd98a411d30013d914bade42194b63c0ab\": rpc error: code = NotFound desc = could not find container \"da50eba50e8926d9ee79c86e17cc10cd98a411d30013d914bade42194b63c0ab\": container with ID starting with da50eba50e8926d9ee79c86e17cc10cd98a411d30013d914bade42194b63c0ab not found: ID does not exist"
Sep 18 18:52:25.605096 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.605072    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn"]
Sep 18 18:52:25.609182 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.609159    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hz7jn"]
Sep 18 18:52:25.724604 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.724568    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64fbd675b6-l9gft"]
Sep 18 18:52:25.727418 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:52:25.727390    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod939e465d_57fd_4592_a8e8_13abfabd5c1a.slice/crio-fed79a382ecf2b34c1aed2f73e434807a1e55c0f73cfa1d7ab5658b1731809a0 WatchSource:0}: Error finding container fed79a382ecf2b34c1aed2f73e434807a1e55c0f73cfa1d7ab5658b1731809a0: Status 404 returned error can't find the container with id fed79a382ecf2b34c1aed2f73e434807a1e55c0f73cfa1d7ab5658b1731809a0
Sep 18 18:52:25.844039 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.844013    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5589a623-805c-4e87-bd9c-8395aa27c58e" path="/var/lib/kubelet/pods/5589a623-805c-4e87-bd9c-8395aa27c58e/volumes"
Sep 18 18:52:25.844389 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:25.844377    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c20c633b-cb18-4a28-a1ef-790869c5893c" path="/var/lib/kubelet/pods/c20c633b-cb18-4a28-a1ef-790869c5893c/volumes"
Sep 18 18:52:26.097302 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:26.097284    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 18 18:52:26.585258 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:26.585221    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fbd675b6-l9gft" event={"ID":"939e465d-57fd-4592-a8e8-13abfabd5c1a","Type":"ContainerStarted","Data":"269dbb2548e2f7ca18e30ff775d1ee188ae896b7a347bfe7fc8ff04c897c7dd8"}
Sep 18 18:52:26.585628 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:26.585278    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fbd675b6-l9gft" event={"ID":"939e465d-57fd-4592-a8e8-13abfabd5c1a","Type":"ContainerStarted","Data":"fed79a382ecf2b34c1aed2f73e434807a1e55c0f73cfa1d7ab5658b1731809a0"}
Sep 18 18:52:26.585628 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:26.585427    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64fbd675b6-l9gft"
Sep 18 18:52:26.602867 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:26.602798    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64fbd675b6-l9gft" podStartSLOduration=1.237270877 podStartE2EDuration="1.602784424s" totalImagesPullingTime="365.513547ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:52:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:52:26.60155375 +0000 UTC m=+782.800037972" watchObservedRunningTime="2026-09-18 18:52:26.602784424 +0000 UTC m=+782.801268628"
Sep 18 18:52:37.591990 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:37.591917    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64fbd675b6-l9gft"
Sep 18 18:52:46.472414 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:46.472375    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:52:46.562101 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:52:46.562071    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:08.004910 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:08.004873    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:08.331151 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:08.331067    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:09.458858 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:09.458794    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:09.548072 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:09.548039    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:10.481115 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:10.481071    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:10.578621 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:10.578581    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:24.681188 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:24.681155    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:24.876139 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:24.876085    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:25.028125 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:25.028092    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:25.132410 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:25.132360    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:25.227483 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:25.227447    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:25.300810 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:25.300729    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:25.421789 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:25.421752    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:25.546411 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:25.546372    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:26.026336 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:26.026299    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:26.122098 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:26.122065    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:26.444554 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:26.444479    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:26.633177 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:26.633142    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8962-feb45d"]
Sep 18 18:53:26.636636 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:26.636621    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8962-feb45d"
Sep 18 18:53:26.639466 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:26.639436    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7m4wl\""
Sep 18 18:53:26.640813 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:26.640794    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8962-feb45d"]
Sep 18 18:53:26.708643 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:26.708571    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvqzm\" (UniqueName: \"kubernetes.io/projected/7e13671d-020b-4b0b-a4b0-0e8f6448ae7c-kube-api-access-pvqzm\") pod \"test-curl-8962-feb45d\" (UID: \"7e13671d-020b-4b0b-a4b0-0e8f6448ae7c\") " pod="opendatahub/test-curl-8962-feb45d"
Sep 18 18:53:26.809955 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:26.809914    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvqzm\" (UniqueName: \"kubernetes.io/projected/7e13671d-020b-4b0b-a4b0-0e8f6448ae7c-kube-api-access-pvqzm\") pod \"test-curl-8962-feb45d\" (UID: \"7e13671d-020b-4b0b-a4b0-0e8f6448ae7c\") " pod="opendatahub/test-curl-8962-feb45d"
Sep 18 18:53:26.818386 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:26.818355    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvqzm\" (UniqueName: \"kubernetes.io/projected/7e13671d-020b-4b0b-a4b0-0e8f6448ae7c-kube-api-access-pvqzm\") pod \"test-curl-8962-feb45d\" (UID: \"7e13671d-020b-4b0b-a4b0-0e8f6448ae7c\") " pod="opendatahub/test-curl-8962-feb45d"
Sep 18 18:53:26.947641 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:26.947614    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8962-feb45d"
Sep 18 18:53:27.223103 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:27.217278    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8962-feb45d"]
Sep 18 18:53:27.892217 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:27.892183    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8962-feb45d" event={"ID":"7e13671d-020b-4b0b-a4b0-0e8f6448ae7c","Type":"ContainerStarted","Data":"f04fa5be9671712ec2385ba32cf89c13eb8c94a8f853ff7bdb8ffcf4b7ec29fe"}
Sep 18 18:53:27.977069 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:27.977036    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:28.077879 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:28.077845    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:28.176589 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:28.176494    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:28.333177 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:28.333141    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:29.128929 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.128896    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"]
Sep 18 18:53:29.138462 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.138438    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 18:53:29.145694 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.142339    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-cc289c-serving-cert\""
Sep 18 18:53:29.145694 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.144801    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"]
Sep 18 18:53:29.177010 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.176973    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:29.233384 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.233328    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-maas-api-tls\") pod \"maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl\" (UID: \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 18:53:29.233567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.233389    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rmjb\" (UniqueName: \"kubernetes.io/projected/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-kube-api-access-4rmjb\") pod \"maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl\" (UID: \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 18:53:29.233567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.233477    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-metrics-certs\") pod \"maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl\" (UID: \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 18:53:29.233567 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.233524    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl\" (UID: \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 18:53:29.276986 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.276906    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:29.334873 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.334839    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-maas-api-tls\") pod \"maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl\" (UID: \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 18:53:29.334873 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.334877    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rmjb\" (UniqueName: \"kubernetes.io/projected/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-kube-api-access-4rmjb\") pod \"maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl\" (UID: \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 18:53:29.335421 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.334908    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-metrics-certs\") pod \"maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl\" (UID: \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 18:53:29.335421 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.334944    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl\" (UID: \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 18:53:29.335421 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:53:29.335005    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-serving-cert: secret "maas-api-e2e-worker-w0-cc289c-serving-cert" not found
Sep 18 18:53:29.335421 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:53:29.335085    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-maas-api-tls podName:1aee097c-ece3-4f75-b8bf-e4043bc9ce20 nodeName:}" failed. No retries permitted until 2026-09-18 18:53:29.835065075 +0000 UTC m=+846.033549264 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-maas-api-tls") pod "maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl" (UID: "1aee097c-ece3-4f75-b8bf-e4043bc9ce20") : secret "maas-api-e2e-worker-w0-cc289c-serving-cert" not found
Sep 18 18:53:29.335704 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.335680    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl\" (UID: \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 18:53:29.338248 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.337644    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-metrics-certs\") pod \"maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl\" (UID: \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 18:53:29.344079 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.344056    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rmjb\" (UniqueName: \"kubernetes.io/projected/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-kube-api-access-4rmjb\") pod \"maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl\" (UID: \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 18:53:29.612008 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.611978    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:29.796699 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.796668    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:29.840106 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.840023    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-maas-api-tls\") pod \"maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl\" (UID: \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 18:53:29.842543 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.842520    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-maas-api-tls\") pod \"maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl\" (UID: \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 18:53:29.892614 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.892574    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:29.903108 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.903081    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8962-feb45d" event={"ID":"7e13671d-020b-4b0b-a4b0-0e8f6448ae7c","Type":"ContainerStarted","Data":"3dbad3edf7db9418e2d36ea85ddc7653777d913de6bd8084de21884b1453dcdf"}
Sep 18 18:53:29.908471 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.908452    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt"]
Sep 18 18:53:29.911932 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.911916    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt"
Sep 18 18:53:29.915006 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.914988    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-cc289c-dockercfg-q54pf\""
Sep 18 18:53:29.915472 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.915458    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-cc289c\""
Sep 18 18:53:29.922016 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.921994    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt"]
Sep 18 18:53:29.927371 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.927350    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr"]
Sep 18 18:53:29.928546 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.928511    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8962-feb45d" podStartSLOduration=1.5794361989999999 podStartE2EDuration="3.928498803s" totalImagesPullingTime="2.349062604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:29.923604488 +0000 UTC m=+846.122088692" watchObservedRunningTime="2026-09-18 18:53:29.928498803 +0000 UTC m=+846.126983009"
Sep 18 18:53:29.931401 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.931383    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr"
Sep 18 18:53:29.940548 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.940529    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ada18349-7cde-43b4-b732-9ec5ac8f0235-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt\" (UID: \"ada18349-7cde-43b4-b732-9ec5ac8f0235\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt"
Sep 18 18:53:29.940665 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.940597    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpczf\" (UniqueName: \"kubernetes.io/projected/ada18349-7cde-43b4-b732-9ec5ac8f0235-kube-api-access-hpczf\") pod \"payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt\" (UID: \"ada18349-7cde-43b4-b732-9ec5ac8f0235\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt"
Sep 18 18:53:29.944081 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:29.944063    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr"]
Sep 18 18:53:30.041785 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.041757    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a35a8ead-e3e9-4ce4-a68d-aa78e257730c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr\" (UID: \"a35a8ead-e3e9-4ce4-a68d-aa78e257730c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr"
Sep 18 18:53:30.041995 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.041865    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ada18349-7cde-43b4-b732-9ec5ac8f0235-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt\" (UID: \"ada18349-7cde-43b4-b732-9ec5ac8f0235\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt"
Sep 18 18:53:30.041995 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.041905    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29nkf\" (UniqueName: \"kubernetes.io/projected/a35a8ead-e3e9-4ce4-a68d-aa78e257730c-kube-api-access-29nkf\") pod \"payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr\" (UID: \"a35a8ead-e3e9-4ce4-a68d-aa78e257730c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr"
Sep 18 18:53:30.041995 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.041939    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpczf\" (UniqueName: \"kubernetes.io/projected/ada18349-7cde-43b4-b732-9ec5ac8f0235-kube-api-access-hpczf\") pod \"payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt\" (UID: \"ada18349-7cde-43b4-b732-9ec5ac8f0235\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt"
Sep 18 18:53:30.042592 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.042571    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ada18349-7cde-43b4-b732-9ec5ac8f0235-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt\" (UID: \"ada18349-7cde-43b4-b732-9ec5ac8f0235\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt"
Sep 18 18:53:30.051379 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.051353    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpczf\" (UniqueName: \"kubernetes.io/projected/ada18349-7cde-43b4-b732-9ec5ac8f0235-kube-api-access-hpczf\") pod \"payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt\" (UID: \"ada18349-7cde-43b4-b732-9ec5ac8f0235\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt"
Sep 18 18:53:30.054080 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.054063    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 18:53:30.142490 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.142452    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a35a8ead-e3e9-4ce4-a68d-aa78e257730c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr\" (UID: \"a35a8ead-e3e9-4ce4-a68d-aa78e257730c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr"
Sep 18 18:53:30.142659 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.142578    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29nkf\" (UniqueName: \"kubernetes.io/projected/a35a8ead-e3e9-4ce4-a68d-aa78e257730c-kube-api-access-29nkf\") pod \"payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr\" (UID: \"a35a8ead-e3e9-4ce4-a68d-aa78e257730c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr"
Sep 18 18:53:30.143175 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.143150    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a35a8ead-e3e9-4ce4-a68d-aa78e257730c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr\" (UID: \"a35a8ead-e3e9-4ce4-a68d-aa78e257730c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr"
Sep 18 18:53:30.151951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.151922    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29nkf\" (UniqueName: \"kubernetes.io/projected/a35a8ead-e3e9-4ce4-a68d-aa78e257730c-kube-api-access-29nkf\") pod \"payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr\" (UID: \"a35a8ead-e3e9-4ce4-a68d-aa78e257730c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr"
Sep 18 18:53:30.223504 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.223474    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt"
Sep 18 18:53:30.240835 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.240804    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr"
Sep 18 18:53:30.340023 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.339998    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"]
Sep 18 18:53:30.341474 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:53:30.341442    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aee097c_ece3_4f75_b8bf_e4043bc9ce20.slice/crio-0f71df8172d06607fee9fb53854fcabdc1110133532de9a38c62655c9db6d04f WatchSource:0}: Error finding container 0f71df8172d06607fee9fb53854fcabdc1110133532de9a38c62655c9db6d04f: Status 404 returned error can't find the container with id 0f71df8172d06607fee9fb53854fcabdc1110133532de9a38c62655c9db6d04f
Sep 18 18:53:30.365168 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.365095    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:30.468599 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.468570    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:30.493465 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.493375    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8962-feb45d"]
Sep 18 18:53:30.495916 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.495896    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8962-feb45d"]
Sep 18 18:53:30.501145 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.501121    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt"]
Sep 18 18:53:30.503689 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:53:30.503092    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podada18349_7cde_43b4_b732_9ec5ac8f0235.slice/crio-0a2a6ee9d00a34e39ee725776e6b8b47308d186d1ee842fe9a04111970d76496 WatchSource:0}: Error finding container 0a2a6ee9d00a34e39ee725776e6b8b47308d186d1ee842fe9a04111970d76496: Status 404 returned error can't find the container with id 0a2a6ee9d00a34e39ee725776e6b8b47308d186d1ee842fe9a04111970d76496
Sep 18 18:53:30.523679 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.523655    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr"]
Sep 18 18:53:30.527586 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:53:30.527560    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda35a8ead_e3e9_4ce4_a68d_aa78e257730c.slice/crio-d9500a86d888ca075d3e66f789b47a7494fcdbe45ffbd5f66f96e47319ba8eca WatchSource:0}: Error finding container d9500a86d888ca075d3e66f789b47a7494fcdbe45ffbd5f66f96e47319ba8eca: Status 404 returned error can't find the container with id d9500a86d888ca075d3e66f789b47a7494fcdbe45ffbd5f66f96e47319ba8eca
Sep 18 18:53:30.681547 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.681505    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:30.854089 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.854054    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:30.909951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.909869    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl" event={"ID":"1aee097c-ece3-4f75-b8bf-e4043bc9ce20","Type":"ContainerStarted","Data":"676e4d773d1d12426c51fc767f680e3f1a202a2431e752bc76d18c2937f29bb9"}
Sep 18 18:53:30.909951 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.909920    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl" event={"ID":"1aee097c-ece3-4f75-b8bf-e4043bc9ce20","Type":"ContainerStarted","Data":"0f71df8172d06607fee9fb53854fcabdc1110133532de9a38c62655c9db6d04f"}
Sep 18 18:53:30.910156 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.909966    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 18:53:30.911702 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.911679    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt" event={"ID":"ada18349-7cde-43b4-b732-9ec5ac8f0235","Type":"ContainerStarted","Data":"5df803e15bf9adfd6035c83ead6395038a8ae476e36eb7339a41e9fc9cead4f6"}
Sep 18 18:53:30.911884 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.911723    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt"
Sep 18 18:53:30.911884 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.911745    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt" event={"ID":"ada18349-7cde-43b4-b732-9ec5ac8f0235","Type":"ContainerStarted","Data":"0a2a6ee9d00a34e39ee725776e6b8b47308d186d1ee842fe9a04111970d76496"}
Sep 18 18:53:30.913381 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.913357    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr" event={"ID":"a35a8ead-e3e9-4ce4-a68d-aa78e257730c","Type":"ContainerStarted","Data":"266e7d2b65d8d2fc12e4e3be6cc4d60b94ef4d9246c3df15a042d07469dccd02"}
Sep 18 18:53:30.913494 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.913399    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr" event={"ID":"a35a8ead-e3e9-4ce4-a68d-aa78e257730c","Type":"ContainerStarted","Data":"d9500a86d888ca075d3e66f789b47a7494fcdbe45ffbd5f66f96e47319ba8eca"}
Sep 18 18:53:30.913547 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:53:30.913497    2558 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8962-feb45d" container="test-curl-8962-feb45d"
Sep 18 18:53:30.929359 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.929312    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl" podStartSLOduration=1.6543959849999998 podStartE2EDuration="1.929300055s" totalImagesPullingTime="274.90407ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:30.927207814 +0000 UTC m=+847.125692042" watchObservedRunningTime="2026-09-18 18:53:30.929300055 +0000 UTC m=+847.127784259"
Sep 18 18:53:30.937460 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.937431    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:30.948897 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.948857    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt" podStartSLOduration=1.9488446160000001 podStartE2EDuration="1.948844616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:30.948646628 +0000 UTC m=+847.147130832" watchObservedRunningTime="2026-09-18 18:53:30.948844616 +0000 UTC m=+847.147328810"
Sep 18 18:53:30.969283 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.969229    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr" podStartSLOduration=1.969211616 podStartE2EDuration="1.969211616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:30.965232665 +0000 UTC m=+847.163716872" watchObservedRunningTime="2026-09-18 18:53:30.969211616 +0000 UTC m=+847.167695823"
Sep 18 18:53:30.994387 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.994350    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66"]
Sep 18 18:53:30.994765 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.994752    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e13671d-020b-4b0b-a4b0-0e8f6448ae7c" containerName="test-curl-8962-feb45d"
Sep 18 18:53:30.994765 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.994766    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e13671d-020b-4b0b-a4b0-0e8f6448ae7c" containerName="test-curl-8962-feb45d"
Sep 18 18:53:30.994897 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.994864    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e13671d-020b-4b0b-a4b0-0e8f6448ae7c" containerName="test-curl-8962-feb45d"
Sep 18 18:53:30.997738 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:30.997720    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66"
Sep 18 18:53:31.000349 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.000329    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-9d9f36-dockercfg-j4rrv\""
Sep 18 18:53:31.000500 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.000369    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-9d9f36\""
Sep 18 18:53:31.006790 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.006768    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66"]
Sep 18 18:53:31.029568 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.029537    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c"]
Sep 18 18:53:31.032948 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.032933    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c"
Sep 18 18:53:31.042258 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.042232    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c"]
Sep 18 18:53:31.051031 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.051006    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/005a4591-36ba-487d-8fbd-5c6e034ab1f0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66\" (UID: \"005a4591-36ba-487d-8fbd-5c6e034ab1f0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66"
Sep 18 18:53:31.051159 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.051135    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnsg4\" (UniqueName: \"kubernetes.io/projected/005a4591-36ba-487d-8fbd-5c6e034ab1f0-kube-api-access-jnsg4\") pod \"payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66\" (UID: \"005a4591-36ba-487d-8fbd-5c6e034ab1f0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66"
Sep 18 18:53:31.152228 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.152190    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02e2b271-43d7-43a2-ae46-991ae54de7bb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c\" (UID: \"02e2b271-43d7-43a2-ae46-991ae54de7bb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c"
Sep 18 18:53:31.152392 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.152247    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnsg4\" (UniqueName: \"kubernetes.io/projected/005a4591-36ba-487d-8fbd-5c6e034ab1f0-kube-api-access-jnsg4\") pod \"payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66\" (UID: \"005a4591-36ba-487d-8fbd-5c6e034ab1f0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66"
Sep 18 18:53:31.152392 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.152354    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk7pq\" (UniqueName: \"kubernetes.io/projected/02e2b271-43d7-43a2-ae46-991ae54de7bb-kube-api-access-bk7pq\") pod \"payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c\" (UID: \"02e2b271-43d7-43a2-ae46-991ae54de7bb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c"
Sep 18 18:53:31.152392 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.152382    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/005a4591-36ba-487d-8fbd-5c6e034ab1f0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66\" (UID: \"005a4591-36ba-487d-8fbd-5c6e034ab1f0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66"
Sep 18 18:53:31.153214 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.153193    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/005a4591-36ba-487d-8fbd-5c6e034ab1f0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66\" (UID: \"005a4591-36ba-487d-8fbd-5c6e034ab1f0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66"
Sep 18 18:53:31.160347 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.160277    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnsg4\" (UniqueName: \"kubernetes.io/projected/005a4591-36ba-487d-8fbd-5c6e034ab1f0-kube-api-access-jnsg4\") pod \"payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66\" (UID: \"005a4591-36ba-487d-8fbd-5c6e034ab1f0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66"
Sep 18 18:53:31.221980 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.221952    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"]
Sep 18 18:53:31.226067 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.226047    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"
Sep 18 18:53:31.228705 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.228687    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-bcb770-serving-cert\""
Sep 18 18:53:31.236404 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.236382    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"]
Sep 18 18:53:31.253663 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.253641    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk7pq\" (UniqueName: \"kubernetes.io/projected/02e2b271-43d7-43a2-ae46-991ae54de7bb-kube-api-access-bk7pq\") pod \"payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c\" (UID: \"02e2b271-43d7-43a2-ae46-991ae54de7bb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c"
Sep 18 18:53:31.253757 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.253697    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02e2b271-43d7-43a2-ae46-991ae54de7bb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c\" (UID: \"02e2b271-43d7-43a2-ae46-991ae54de7bb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c"
Sep 18 18:53:31.254360 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.254339    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02e2b271-43d7-43a2-ae46-991ae54de7bb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c\" (UID: \"02e2b271-43d7-43a2-ae46-991ae54de7bb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c"
Sep 18 18:53:31.265152 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.265131    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk7pq\" (UniqueName: \"kubernetes.io/projected/02e2b271-43d7-43a2-ae46-991ae54de7bb-kube-api-access-bk7pq\") pod \"payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c\" (UID: \"02e2b271-43d7-43a2-ae46-991ae54de7bb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c"
Sep 18 18:53:31.310690 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.310669    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66"
Sep 18 18:53:31.315240 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.315197    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:31.344041 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.344015    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c"
Sep 18 18:53:31.355243 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.355212    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/850a7519-641a-4a10-9509-6bb7f462b0af-openshift-service-ca\") pod \"maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49\" (UID: \"850a7519-641a-4a10-9509-6bb7f462b0af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"
Sep 18 18:53:31.355381 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.355271    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnlhz\" (UniqueName: \"kubernetes.io/projected/850a7519-641a-4a10-9509-6bb7f462b0af-kube-api-access-cnlhz\") pod \"maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49\" (UID: \"850a7519-641a-4a10-9509-6bb7f462b0af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"
Sep 18 18:53:31.355381 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.355347    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/850a7519-641a-4a10-9509-6bb7f462b0af-maas-api-tls\") pod \"maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49\" (UID: \"850a7519-641a-4a10-9509-6bb7f462b0af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"
Sep 18 18:53:31.355511 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.355384    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/850a7519-641a-4a10-9509-6bb7f462b0af-metrics-certs\") pod \"maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49\" (UID: \"850a7519-641a-4a10-9509-6bb7f462b0af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"
Sep 18 18:53:31.456610 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.456538    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/850a7519-641a-4a10-9509-6bb7f462b0af-openshift-service-ca\") pod \"maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49\" (UID: \"850a7519-641a-4a10-9509-6bb7f462b0af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"
Sep 18 18:53:31.456610 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.456587    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnlhz\" (UniqueName: \"kubernetes.io/projected/850a7519-641a-4a10-9509-6bb7f462b0af-kube-api-access-cnlhz\") pod \"maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49\" (UID: \"850a7519-641a-4a10-9509-6bb7f462b0af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"
Sep 18 18:53:31.456780 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.456625    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/850a7519-641a-4a10-9509-6bb7f462b0af-maas-api-tls\") pod \"maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49\" (UID: \"850a7519-641a-4a10-9509-6bb7f462b0af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"
Sep 18 18:53:31.456780 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.456653    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/850a7519-641a-4a10-9509-6bb7f462b0af-metrics-certs\") pod \"maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49\" (UID: \"850a7519-641a-4a10-9509-6bb7f462b0af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"
Sep 18 18:53:31.457178 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.457156    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/850a7519-641a-4a10-9509-6bb7f462b0af-openshift-service-ca\") pod \"maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49\" (UID: \"850a7519-641a-4a10-9509-6bb7f462b0af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"
Sep 18 18:53:31.458910 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.458884    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/850a7519-641a-4a10-9509-6bb7f462b0af-metrics-certs\") pod \"maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49\" (UID: \"850a7519-641a-4a10-9509-6bb7f462b0af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"
Sep 18 18:53:31.459108 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.459088    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/850a7519-641a-4a10-9509-6bb7f462b0af-maas-api-tls\") pod \"maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49\" (UID: \"850a7519-641a-4a10-9509-6bb7f462b0af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"
Sep 18 18:53:31.464577 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.464546    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnlhz\" (UniqueName: \"kubernetes.io/projected/850a7519-641a-4a10-9509-6bb7f462b0af-kube-api-access-cnlhz\") pod \"maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49\" (UID: \"850a7519-641a-4a10-9509-6bb7f462b0af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"
Sep 18 18:53:31.507524 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.507492    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:31.541160 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.541136    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"
Sep 18 18:53:31.592741 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.592235    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66"]
Sep 18 18:53:31.595627 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:53:31.595599    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod005a4591_36ba_487d_8fbd_5c6e034ab1f0.slice/crio-f46256c9948c48dda18413b0a50c51f9bd1e8be1d22b50d1a152c068fcc5737d WatchSource:0}: Error finding container f46256c9948c48dda18413b0a50c51f9bd1e8be1d22b50d1a152c068fcc5737d: Status 404 returned error can't find the container with id f46256c9948c48dda18413b0a50c51f9bd1e8be1d22b50d1a152c068fcc5737d
Sep 18 18:53:31.628504 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.628482    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c"]
Sep 18 18:53:31.636083 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:53:31.636058    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02e2b271_43d7_43a2_ae46_991ae54de7bb.slice/crio-4c51f88462c1de305138cedb46aeca6ff09d49997010bdf04d94c416dc94bc4b WatchSource:0}: Error finding container 4c51f88462c1de305138cedb46aeca6ff09d49997010bdf04d94c416dc94bc4b: Status 404 returned error can't find the container with id 4c51f88462c1de305138cedb46aeca6ff09d49997010bdf04d94c416dc94bc4b
Sep 18 18:53:31.820002 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.819969    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"]
Sep 18 18:53:31.821519 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:53:31.821489    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod850a7519_641a_4a10_9509_6bb7f462b0af.slice/crio-764ec44b48133aea40002ce3dd871569ea767581ed030d7603b9b4eaf0ee4e41 WatchSource:0}: Error finding container 764ec44b48133aea40002ce3dd871569ea767581ed030d7603b9b4eaf0ee4e41: Status 404 returned error can't find the container with id 764ec44b48133aea40002ce3dd871569ea767581ed030d7603b9b4eaf0ee4e41
Sep 18 18:53:31.915056 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.914960    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:31.921327 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.921298    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66" event={"ID":"005a4591-36ba-487d-8fbd-5c6e034ab1f0","Type":"ContainerStarted","Data":"e59c63358a379be2e1003e594538dbea3b7717d40d569342fe9ca80b62da4c1f"}
Sep 18 18:53:31.921483 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.921345    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66" event={"ID":"005a4591-36ba-487d-8fbd-5c6e034ab1f0","Type":"ContainerStarted","Data":"f46256c9948c48dda18413b0a50c51f9bd1e8be1d22b50d1a152c068fcc5737d"}
Sep 18 18:53:31.921483 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.921441    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66"
Sep 18 18:53:31.923235 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.923214    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c" event={"ID":"02e2b271-43d7-43a2-ae46-991ae54de7bb","Type":"ContainerStarted","Data":"284983b2e767641e30dec222eb22d1a3183abf1544b98eabe744f0b0e1f4d3eb"}
Sep 18 18:53:31.923368 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.923254    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c"
Sep 18 18:53:31.923368 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.923270    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c" event={"ID":"02e2b271-43d7-43a2-ae46-991ae54de7bb","Type":"ContainerStarted","Data":"4c51f88462c1de305138cedb46aeca6ff09d49997010bdf04d94c416dc94bc4b"}
Sep 18 18:53:31.924599 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.924549    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49" event={"ID":"850a7519-641a-4a10-9509-6bb7f462b0af","Type":"ContainerStarted","Data":"764ec44b48133aea40002ce3dd871569ea767581ed030d7603b9b4eaf0ee4e41"}
Sep 18 18:53:31.924599 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.924587    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8962-feb45d" podUID="7e13671d-020b-4b0b-a4b0-0e8f6448ae7c" containerName="test-curl-8962-feb45d" containerID="cri-o://3dbad3edf7db9418e2d36ea85ddc7653777d913de6bd8084de21884b1453dcdf" gracePeriod=2
Sep 18 18:53:31.925092 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.925036    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr"
Sep 18 18:53:31.945667 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.945615    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66" podStartSLOduration=1.9456032250000002 podStartE2EDuration="1.945603225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:31.943810535 +0000 UTC m=+848.142294764" watchObservedRunningTime="2026-09-18 18:53:31.945603225 +0000 UTC m=+848.144087428"
Sep 18 18:53:31.968387 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:31.968335    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c" podStartSLOduration=0.968316232 podStartE2EDuration="968.316232ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:31.966570577 +0000 UTC m=+848.165054794" watchObservedRunningTime="2026-09-18 18:53:31.968316232 +0000 UTC m=+848.166800440"
Sep 18 18:53:32.018804 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:32.018767    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:32.347643 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:32.347610    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:32.889683 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:32.889651    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:32.935371 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:32.935333    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49" event={"ID":"850a7519-641a-4a10-9509-6bb7f462b0af","Type":"ContainerStarted","Data":"01faf3d6d12bf2ab55397f17344ed1bcc85800ec6dd365ad66a96d6fc97cbb4b"}
Sep 18 18:53:32.935869 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:32.935850    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"
Sep 18 18:53:32.954529 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:32.954454    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49" podStartSLOduration=1.5489549120000001 podStartE2EDuration="1.954434536s" totalImagesPullingTime="405.479624ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:32.953227135 +0000 UTC m=+849.151711342" watchObservedRunningTime="2026-09-18 18:53:32.954434536 +0000 UTC m=+849.152918741"
Sep 18 18:53:32.958236 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:32.958202    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:33.122359 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:33.122325    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:33.218324 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:33.218238    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:33.437573 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:33.437539    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:34.075188 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:34.075152    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:34.102630 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:34.102607    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8962-feb45d"
Sep 18 18:53:34.167127 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:34.167097    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:34.187663 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:34.187589    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvqzm\" (UniqueName: \"kubernetes.io/projected/7e13671d-020b-4b0b-a4b0-0e8f6448ae7c-kube-api-access-pvqzm\") pod \"7e13671d-020b-4b0b-a4b0-0e8f6448ae7c\" (UID: \"7e13671d-020b-4b0b-a4b0-0e8f6448ae7c\") "
Sep 18 18:53:34.189690 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:34.189667    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e13671d-020b-4b0b-a4b0-0e8f6448ae7c-kube-api-access-pvqzm" (OuterVolumeSpecName: "kube-api-access-pvqzm") pod "7e13671d-020b-4b0b-a4b0-0e8f6448ae7c" (UID: "7e13671d-020b-4b0b-a4b0-0e8f6448ae7c"). InnerVolumeSpecName "kube-api-access-pvqzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:53:34.288680 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:34.288642    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvqzm\" (UniqueName: \"kubernetes.io/projected/7e13671d-020b-4b0b-a4b0-0e8f6448ae7c-kube-api-access-pvqzm\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:53:34.949202 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:34.949158    2558 generic.go:356] "Generic (PLEG): container finished" podID="7e13671d-020b-4b0b-a4b0-0e8f6448ae7c" containerID="3dbad3edf7db9418e2d36ea85ddc7653777d913de6bd8084de21884b1453dcdf" exitCode=137
Sep 18 18:53:34.949202 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:34.949196    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8962-feb45d"
Sep 18 18:53:34.949724 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:34.949256    2558 scope.go:117] "RemoveContainer" containerID="3dbad3edf7db9418e2d36ea85ddc7653777d913de6bd8084de21884b1453dcdf"
Sep 18 18:53:34.959543 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:34.959518    2558 scope.go:117] "RemoveContainer" containerID="3dbad3edf7db9418e2d36ea85ddc7653777d913de6bd8084de21884b1453dcdf"
Sep 18 18:53:34.959891 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:53:34.959864    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dbad3edf7db9418e2d36ea85ddc7653777d913de6bd8084de21884b1453dcdf\": container with ID starting with 3dbad3edf7db9418e2d36ea85ddc7653777d913de6bd8084de21884b1453dcdf not found: ID does not exist" containerID="3dbad3edf7db9418e2d36ea85ddc7653777d913de6bd8084de21884b1453dcdf"
Sep 18 18:53:34.960002 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:34.959900    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dbad3edf7db9418e2d36ea85ddc7653777d913de6bd8084de21884b1453dcdf"} err="failed to get container status \"3dbad3edf7db9418e2d36ea85ddc7653777d913de6bd8084de21884b1453dcdf\": rpc error: code = NotFound desc = could not find container \"3dbad3edf7db9418e2d36ea85ddc7653777d913de6bd8084de21884b1453dcdf\": container with ID starting with 3dbad3edf7db9418e2d36ea85ddc7653777d913de6bd8084de21884b1453dcdf not found: ID does not exist"
Sep 18 18:53:34.960196 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:53:34.960174    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8962-feb45d\" is forbidden: User \"system:node:ip-10-0-130-118.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-118.ec2.internal' and this object" podUID="7e13671d-020b-4b0b-a4b0-0e8f6448ae7c" pod="opendatahub/test-curl-8962-feb45d"
Sep 18 18:53:35.045935 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:35.045894    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:35.650508 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:35.650472    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:35.748367 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:35.748332    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:35.842741 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:53:35.842699    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8962-feb45d\" is forbidden: User \"system:node:ip-10-0-130-118.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-118.ec2.internal' and this object" podUID="7e13671d-020b-4b0b-a4b0-0e8f6448ae7c" pod="opendatahub/test-curl-8962-feb45d"
Sep 18 18:53:35.846597 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:35.846572    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e13671d-020b-4b0b-a4b0-0e8f6448ae7c" path="/var/lib/kubelet/pods/7e13671d-020b-4b0b-a4b0-0e8f6448ae7c/volumes"
Sep 18 18:53:35.953511 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:35.953423    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:36.837347 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:36.837314    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:36.930557 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:36.930532    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 18:53:37.744159 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:37.744130    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:38.397662 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:38.397624    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"]
Sep 18 18:53:38.402681 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:38.402660    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:53:38.404996 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:38.404979    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-6cc65f16-serving-cert\""
Sep 18 18:53:38.412852 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:38.412810    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"]
Sep 18 18:53:38.433484 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:38.433453    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:38.531110 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:38.531081    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c30e6c8e-5965-498f-820c-76ec1113d5fe-metrics-certs\") pod \"maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp\" (UID: \"c30e6c8e-5965-498f-820c-76ec1113d5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:53:38.531276 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:38.531164    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29jgj\" (UniqueName: \"kubernetes.io/projected/c30e6c8e-5965-498f-820c-76ec1113d5fe-kube-api-access-29jgj\") pod \"maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp\" (UID: \"c30e6c8e-5965-498f-820c-76ec1113d5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:53:38.531276 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:38.531210    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c30e6c8e-5965-498f-820c-76ec1113d5fe-maas-api-tls\") pod \"maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp\" (UID: \"c30e6c8e-5965-498f-820c-76ec1113d5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:53:38.531276 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:38.531228    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c30e6c8e-5965-498f-820c-76ec1113d5fe-openshift-service-ca\") pod \"maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp\" (UID: \"c30e6c8e-5965-498f-820c-76ec1113d5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:53:38.531585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:38.531566    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:38.632042 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:38.632013    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29jgj\" (UniqueName: \"kubernetes.io/projected/c30e6c8e-5965-498f-820c-76ec1113d5fe-kube-api-access-29jgj\") pod \"maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp\" (UID: \"c30e6c8e-5965-498f-820c-76ec1113d5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:53:38.632210 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:38.632059    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c30e6c8e-5965-498f-820c-76ec1113d5fe-maas-api-tls\") pod \"maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp\" (UID: \"c30e6c8e-5965-498f-820c-76ec1113d5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:53:38.632210 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:38.632131    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c30e6c8e-5965-498f-820c-76ec1113d5fe-openshift-service-ca\") pod \"maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp\" (UID: \"c30e6c8e-5965-498f-820c-76ec1113d5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:53:38.632329 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:53:38.632217    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-serving-cert: secret "maas-api-e2e-ait-6cc65f16-serving-cert" not found
Sep 18 18:53:38.632329 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:38.632261    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c30e6c8e-5965-498f-820c-76ec1113d5fe-metrics-certs\") pod \"maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp\" (UID: \"c30e6c8e-5965-498f-820c-76ec1113d5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:53:38.632329 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:53:38.632273    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c30e6c8e-5965-498f-820c-76ec1113d5fe-maas-api-tls podName:c30e6c8e-5965-498f-820c-76ec1113d5fe nodeName:}" failed. No retries permitted until 2026-09-18 18:53:39.132257129 +0000 UTC m=+855.330741310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c30e6c8e-5965-498f-820c-76ec1113d5fe-maas-api-tls") pod "maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp" (UID: "c30e6c8e-5965-498f-820c-76ec1113d5fe") : secret "maas-api-e2e-ait-6cc65f16-serving-cert" not found
Sep 18 18:53:38.632872 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:38.632855    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c30e6c8e-5965-498f-820c-76ec1113d5fe-openshift-service-ca\") pod \"maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp\" (UID: \"c30e6c8e-5965-498f-820c-76ec1113d5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:53:38.634524 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:38.634504    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c30e6c8e-5965-498f-820c-76ec1113d5fe-metrics-certs\") pod \"maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp\" (UID: \"c30e6c8e-5965-498f-820c-76ec1113d5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:53:38.642684 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:38.642660    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29jgj\" (UniqueName: \"kubernetes.io/projected/c30e6c8e-5965-498f-820c-76ec1113d5fe-kube-api-access-29jgj\") pod \"maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp\" (UID: \"c30e6c8e-5965-498f-820c-76ec1113d5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:53:38.922498 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:38.922466    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:39.018263 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:39.018234    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:39.101880 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:39.101846    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:39.137155 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:39.137126    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c30e6c8e-5965-498f-820c-76ec1113d5fe-maas-api-tls\") pod \"maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp\" (UID: \"c30e6c8e-5965-498f-820c-76ec1113d5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:53:39.139590 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:39.139565    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c30e6c8e-5965-498f-820c-76ec1113d5fe-maas-api-tls\") pod \"maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp\" (UID: \"c30e6c8e-5965-498f-820c-76ec1113d5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:53:39.314388 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:39.314357    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:53:39.588885 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:39.588857    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"]
Sep 18 18:53:39.590460 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:53:39.590437    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc30e6c8e_5965_498f_820c_76ec1113d5fe.slice/crio-286c131b8a64bb5bc3a453b27504dcfca63dcdfb836cb4de7563c8d288134266 WatchSource:0}: Error finding container 286c131b8a64bb5bc3a453b27504dcfca63dcdfb836cb4de7563c8d288134266: Status 404 returned error can't find the container with id 286c131b8a64bb5bc3a453b27504dcfca63dcdfb836cb4de7563c8d288134266
Sep 18 18:53:39.954459 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:39.954437    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"
Sep 18 18:53:39.981117 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:39.981086    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp" event={"ID":"c30e6c8e-5965-498f-820c-76ec1113d5fe","Type":"ContainerStarted","Data":"286c131b8a64bb5bc3a453b27504dcfca63dcdfb836cb4de7563c8d288134266"}
Sep 18 18:53:40.987112 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:40.987079    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp" event={"ID":"c30e6c8e-5965-498f-820c-76ec1113d5fe","Type":"ContainerStarted","Data":"1c3082dea0fdad00014c0449f6b3eb65a03111a2d8dac80d3a6eda78a257cd53"}
Sep 18 18:53:40.987502 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:40.987218    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:53:41.005237 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:41.005193    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp" podStartSLOduration=2.68769488 podStartE2EDuration="3.005179648s" totalImagesPullingTime="317.484768ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:41.003617218 +0000 UTC m=+857.202101443" watchObservedRunningTime="2026-09-18 18:53:41.005179648 +0000 UTC m=+857.203663852"
Sep 18 18:53:41.813994 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:41.813964    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:41.927296 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:41.927268    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt"
Sep 18 18:53:42.937000 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:42.936971    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66"
Sep 18 18:53:42.937479 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:42.937342    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr"
Sep 18 18:53:42.937969 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:42.937947    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c"
Sep 18 18:53:43.425420 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:43.425384    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:43.528123 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:43.528080    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:43.634182 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:43.634157    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:46.998260 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:46.998233    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:53:49.717367 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:49.717333    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:49.838644 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:49.838593    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:49.930363 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:49.930320    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:50.068778 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:50.068740    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:50.222328 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:50.222290    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:51.789917 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:51.789879    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:52.583401 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:52.583356    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:52.872926 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:52.872810    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:52.970867 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:52.970835    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:53.315487 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:53.315455    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:53.429255 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:53.429222    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:53.529763 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:53.529727    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:54.448342 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:54.448311    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:55.296061 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:55.296029    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:56.235308 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:56.235267    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:56.406609 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:56.406573    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:56.506677 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:56.506650    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:57.016926 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:57.016894    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:57.687184 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:57.687150    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:58.020779 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:58.020740    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:58.347335 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:58.347221    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:58.447433 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:58.447399    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:58.612098 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:58.612015    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:58.853983 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:58.853942    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:53:59.340572 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:53:59.340517    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:00.032150 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:00.032112    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:00.715156 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:00.715119    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:01.599217 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:01.599184    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:02.156650 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:02.156614    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:02.278248 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:02.278202    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:02.371463 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:02.371424    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:03.034426 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:03.034396    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:03.927230 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:03.927193    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:04.625321 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:04.625283    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:05.286685 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:05.286652    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:05.949939 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:05.949905    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:06.010775 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:06.010743    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:06.123779 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:06.123752    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:06.411721 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:06.411686    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:06.510847 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:06.510802    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:06.518734 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:06.518709    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn"]
Sep 18 18:54:06.524639 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:06.524620    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn"
Sep 18 18:54:06.527480 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:06.527460    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-897f61-dockercfg-xtcfz\""
Sep 18 18:54:06.527779 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:06.527487    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-897f61\""
Sep 18 18:54:06.530897 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:06.530877    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn"]
Sep 18 18:54:06.600010 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:06.599974    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq8sn\" (UniqueName: \"kubernetes.io/projected/e54405a6-40d3-4ac3-a42e-3c33fc8ffffa-kube-api-access-wq8sn\") pod \"payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn\" (UID: \"e54405a6-40d3-4ac3-a42e-3c33fc8ffffa\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn"
Sep 18 18:54:06.600010 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:06.600014    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e54405a6-40d3-4ac3-a42e-3c33fc8ffffa-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn\" (UID: \"e54405a6-40d3-4ac3-a42e-3c33fc8ffffa\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn"
Sep 18 18:54:06.701612 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:06.701535    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq8sn\" (UniqueName: \"kubernetes.io/projected/e54405a6-40d3-4ac3-a42e-3c33fc8ffffa-kube-api-access-wq8sn\") pod \"payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn\" (UID: \"e54405a6-40d3-4ac3-a42e-3c33fc8ffffa\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn"
Sep 18 18:54:06.701612 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:06.701572    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e54405a6-40d3-4ac3-a42e-3c33fc8ffffa-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn\" (UID: \"e54405a6-40d3-4ac3-a42e-3c33fc8ffffa\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn"
Sep 18 18:54:06.702358 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:06.702333    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e54405a6-40d3-4ac3-a42e-3c33fc8ffffa-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn\" (UID: \"e54405a6-40d3-4ac3-a42e-3c33fc8ffffa\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn"
Sep 18 18:54:06.709798 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:06.709776    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq8sn\" (UniqueName: \"kubernetes.io/projected/e54405a6-40d3-4ac3-a42e-3c33fc8ffffa-kube-api-access-wq8sn\") pod \"payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn\" (UID: \"e54405a6-40d3-4ac3-a42e-3c33fc8ffffa\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn"
Sep 18 18:54:06.846118 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:06.846081    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:06.870961 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:06.870937    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn"
Sep 18 18:54:07.145310 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.145261    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn"]
Sep 18 18:54:07.147711 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:54:07.147685    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode54405a6_40d3_4ac3_a42e_3c33fc8ffffa.slice/crio-ebf90464bbdcd96661ffeee281a1f20a369e88457102373eb78bf1e2f4316177 WatchSource:0}: Error finding container ebf90464bbdcd96661ffeee281a1f20a369e88457102373eb78bf1e2f4316177: Status 404 returned error can't find the container with id ebf90464bbdcd96661ffeee281a1f20a369e88457102373eb78bf1e2f4316177
Sep 18 18:54:07.467029 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.462528    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"]
Sep 18 18:54:07.467029 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.462925    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp" podUID="c30e6c8e-5965-498f-820c-76ec1113d5fe" containerName="maas-api" containerID="cri-o://1c3082dea0fdad00014c0449f6b3eb65a03111a2d8dac80d3a6eda78a257cd53" gracePeriod=30
Sep 18 18:54:07.490234 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.490175    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:07.591585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.591541    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:07.690145 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.690109    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:07.737596 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.737542    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:54:07.761546 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.761458    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:07.813240 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.813213    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c30e6c8e-5965-498f-820c-76ec1113d5fe-metrics-certs\") pod \"c30e6c8e-5965-498f-820c-76ec1113d5fe\" (UID: \"c30e6c8e-5965-498f-820c-76ec1113d5fe\") "
Sep 18 18:54:07.813431 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.813267    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c30e6c8e-5965-498f-820c-76ec1113d5fe-maas-api-tls\") pod \"c30e6c8e-5965-498f-820c-76ec1113d5fe\" (UID: \"c30e6c8e-5965-498f-820c-76ec1113d5fe\") "
Sep 18 18:54:07.813431 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.813381    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c30e6c8e-5965-498f-820c-76ec1113d5fe-openshift-service-ca\") pod \"c30e6c8e-5965-498f-820c-76ec1113d5fe\" (UID: \"c30e6c8e-5965-498f-820c-76ec1113d5fe\") "
Sep 18 18:54:07.813776 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.813749    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c30e6c8e-5965-498f-820c-76ec1113d5fe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c30e6c8e-5965-498f-820c-76ec1113d5fe" (UID: "c30e6c8e-5965-498f-820c-76ec1113d5fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:54:07.813909 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.813873    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29jgj\" (UniqueName: \"kubernetes.io/projected/c30e6c8e-5965-498f-820c-76ec1113d5fe-kube-api-access-29jgj\") pod \"c30e6c8e-5965-498f-820c-76ec1113d5fe\" (UID: \"c30e6c8e-5965-498f-820c-76ec1113d5fe\") "
Sep 18 18:54:07.814258 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.814236    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c30e6c8e-5965-498f-820c-76ec1113d5fe-openshift-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:54:07.817799 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.817775    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c30e6c8e-5965-498f-820c-76ec1113d5fe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c30e6c8e-5965-498f-820c-76ec1113d5fe" (UID: "c30e6c8e-5965-498f-820c-76ec1113d5fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:54:07.819153 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.819131    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c30e6c8e-5965-498f-820c-76ec1113d5fe-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c30e6c8e-5965-498f-820c-76ec1113d5fe" (UID: "c30e6c8e-5965-498f-820c-76ec1113d5fe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:54:07.819276 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.819201    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c30e6c8e-5965-498f-820c-76ec1113d5fe-kube-api-access-29jgj" (OuterVolumeSpecName: "kube-api-access-29jgj") pod "c30e6c8e-5965-498f-820c-76ec1113d5fe" (UID: "c30e6c8e-5965-498f-820c-76ec1113d5fe"). InnerVolumeSpecName "kube-api-access-29jgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:54:07.848550 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.848478    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:07.915582 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.915551    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29jgj\" (UniqueName: \"kubernetes.io/projected/c30e6c8e-5965-498f-820c-76ec1113d5fe-kube-api-access-29jgj\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:54:07.915582 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.915584    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c30e6c8e-5965-498f-820c-76ec1113d5fe-metrics-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:54:07.915801 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:07.915598    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c30e6c8e-5965-498f-820c-76ec1113d5fe-maas-api-tls\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:54:08.133146 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:08.132634    2558 generic.go:356] "Generic (PLEG): container finished" podID="c30e6c8e-5965-498f-820c-76ec1113d5fe" containerID="1c3082dea0fdad00014c0449f6b3eb65a03111a2d8dac80d3a6eda78a257cd53" exitCode=0
Sep 18 18:54:08.133146 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:08.132765    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp" event={"ID":"c30e6c8e-5965-498f-820c-76ec1113d5fe","Type":"ContainerDied","Data":"1c3082dea0fdad00014c0449f6b3eb65a03111a2d8dac80d3a6eda78a257cd53"}
Sep 18 18:54:08.133146 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:08.132812    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp" event={"ID":"c30e6c8e-5965-498f-820c-76ec1113d5fe","Type":"ContainerDied","Data":"286c131b8a64bb5bc3a453b27504dcfca63dcdfb836cb4de7563c8d288134266"}
Sep 18 18:54:08.133146 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:08.132879    2558 scope.go:117] "RemoveContainer" containerID="1c3082dea0fdad00014c0449f6b3eb65a03111a2d8dac80d3a6eda78a257cd53"
Sep 18 18:54:08.133146 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:08.133020    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"
Sep 18 18:54:08.136764 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:08.136737    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn" event={"ID":"e54405a6-40d3-4ac3-a42e-3c33fc8ffffa","Type":"ContainerStarted","Data":"888da155b261bbbc492825b27b0a00e95db3863ae5f12b0147d215da3a96d42b"}
Sep 18 18:54:08.136878 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:08.136783    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn" event={"ID":"e54405a6-40d3-4ac3-a42e-3c33fc8ffffa","Type":"ContainerStarted","Data":"ebf90464bbdcd96661ffeee281a1f20a369e88457102373eb78bf1e2f4316177"}
Sep 18 18:54:08.136878 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:08.136804    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn"
Sep 18 18:54:08.143574 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:08.143557    2558 scope.go:117] "RemoveContainer" containerID="1c3082dea0fdad00014c0449f6b3eb65a03111a2d8dac80d3a6eda78a257cd53"
Sep 18 18:54:08.143930 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:54:08.143814    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c3082dea0fdad00014c0449f6b3eb65a03111a2d8dac80d3a6eda78a257cd53\": container with ID starting with 1c3082dea0fdad00014c0449f6b3eb65a03111a2d8dac80d3a6eda78a257cd53 not found: ID does not exist" containerID="1c3082dea0fdad00014c0449f6b3eb65a03111a2d8dac80d3a6eda78a257cd53"
Sep 18 18:54:08.143999 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:08.143940    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c3082dea0fdad00014c0449f6b3eb65a03111a2d8dac80d3a6eda78a257cd53"} err="failed to get container status \"1c3082dea0fdad00014c0449f6b3eb65a03111a2d8dac80d3a6eda78a257cd53\": rpc error: code = NotFound desc = could not find container \"1c3082dea0fdad00014c0449f6b3eb65a03111a2d8dac80d3a6eda78a257cd53\": container with ID starting with 1c3082dea0fdad00014c0449f6b3eb65a03111a2d8dac80d3a6eda78a257cd53 not found: ID does not exist"
Sep 18 18:54:08.161878 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:08.161856    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"]
Sep 18 18:54:08.169940 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:08.169915    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6cc65f16-7f4c984b88-2cwxp"]
Sep 18 18:54:08.196071 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:08.196031    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn" podStartSLOduration=2.196017934 podStartE2EDuration="2.196017934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:54:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:54:08.194741377 +0000 UTC m=+884.393225597" watchObservedRunningTime="2026-09-18 18:54:08.196017934 +0000 UTC m=+884.394502158"
Sep 18 18:54:08.336861 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:08.336802    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:08.424288 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:08.424213    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:08.508106 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:08.508068    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:08.789447 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:08.789411    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:08.869238 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:08.869200    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:09.057616 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:09.057528    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:09.648012 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:09.647975    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:09.845218 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:09.845187    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c30e6c8e-5965-498f-820c-76ec1113d5fe" path="/var/lib/kubelet/pods/c30e6c8e-5965-498f-820c-76ec1113d5fe/volumes"
Sep 18 18:54:10.433334 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:10.433303    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:10.937134 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:10.937098    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:11.035833 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:11.035781    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:11.311161 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:11.311117    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:12.000166 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.000136    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:12.705549 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.705516    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"]
Sep 18 18:54:12.706018 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.705999    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c30e6c8e-5965-498f-820c-76ec1113d5fe" containerName="maas-api"
Sep 18 18:54:12.706018 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.706016    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c30e6c8e-5965-498f-820c-76ec1113d5fe" containerName="maas-api"
Sep 18 18:54:12.706203 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.706118    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c30e6c8e-5965-498f-820c-76ec1113d5fe" containerName="maas-api"
Sep 18 18:54:12.709310 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.709292    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:12.711898 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.711873    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-e8c645ee-serving-cert\""
Sep 18 18:54:12.720875 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.720851    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"]
Sep 18 18:54:12.744737 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.744696    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:12.762494 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.762466    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhqtm\" (UniqueName: \"kubernetes.io/projected/6e344933-c927-4f6f-8203-7e3cf2028de5-kube-api-access-fhqtm\") pod \"maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6\" (UID: \"6e344933-c927-4f6f-8203-7e3cf2028de5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:12.762632 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.762525    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e344933-c927-4f6f-8203-7e3cf2028de5-openshift-service-ca\") pod \"maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6\" (UID: \"6e344933-c927-4f6f-8203-7e3cf2028de5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:12.762632 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.762598    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e344933-c927-4f6f-8203-7e3cf2028de5-metrics-certs\") pod \"maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6\" (UID: \"6e344933-c927-4f6f-8203-7e3cf2028de5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:12.762760 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.762702    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e344933-c927-4f6f-8203-7e3cf2028de5-maas-api-tls\") pod \"maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6\" (UID: \"6e344933-c927-4f6f-8203-7e3cf2028de5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:12.841140 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.841107    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:12.863207 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.863176    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e344933-c927-4f6f-8203-7e3cf2028de5-openshift-service-ca\") pod \"maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6\" (UID: \"6e344933-c927-4f6f-8203-7e3cf2028de5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:12.863317 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.863223    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e344933-c927-4f6f-8203-7e3cf2028de5-metrics-certs\") pod \"maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6\" (UID: \"6e344933-c927-4f6f-8203-7e3cf2028de5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:12.863317 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.863295    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e344933-c927-4f6f-8203-7e3cf2028de5-maas-api-tls\") pod \"maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6\" (UID: \"6e344933-c927-4f6f-8203-7e3cf2028de5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:12.863442 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.863351    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhqtm\" (UniqueName: \"kubernetes.io/projected/6e344933-c927-4f6f-8203-7e3cf2028de5-kube-api-access-fhqtm\") pod \"maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6\" (UID: \"6e344933-c927-4f6f-8203-7e3cf2028de5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:12.863505 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:54:12.863457    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-serving-cert: secret "maas-api-e2e-migrate-e8c645ee-serving-cert" not found
Sep 18 18:54:12.863565 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:54:12.863517    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e344933-c927-4f6f-8203-7e3cf2028de5-maas-api-tls podName:6e344933-c927-4f6f-8203-7e3cf2028de5 nodeName:}" failed. No retries permitted until 2026-09-18 18:54:13.363503186 +0000 UTC m=+889.561987368 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6e344933-c927-4f6f-8203-7e3cf2028de5-maas-api-tls") pod "maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6" (UID: "6e344933-c927-4f6f-8203-7e3cf2028de5") : secret "maas-api-e2e-migrate-e8c645ee-serving-cert" not found
Sep 18 18:54:12.863811 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.863792    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e344933-c927-4f6f-8203-7e3cf2028de5-openshift-service-ca\") pod \"maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6\" (UID: \"6e344933-c927-4f6f-8203-7e3cf2028de5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:12.865631 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.865610    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e344933-c927-4f6f-8203-7e3cf2028de5-metrics-certs\") pod \"maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6\" (UID: \"6e344933-c927-4f6f-8203-7e3cf2028de5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:12.872161 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.872137    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhqtm\" (UniqueName: \"kubernetes.io/projected/6e344933-c927-4f6f-8203-7e3cf2028de5-kube-api-access-fhqtm\") pod \"maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6\" (UID: \"6e344933-c927-4f6f-8203-7e3cf2028de5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:12.987497 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:12.987388    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:13.365178 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:13.365138    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:13.369290 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:13.369257    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e344933-c927-4f6f-8203-7e3cf2028de5-maas-api-tls\") pod \"maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6\" (UID: \"6e344933-c927-4f6f-8203-7e3cf2028de5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:13.372275 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:13.372243    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e344933-c927-4f6f-8203-7e3cf2028de5-maas-api-tls\") pod \"maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6\" (UID: \"6e344933-c927-4f6f-8203-7e3cf2028de5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:13.453508 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:13.453464    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:13.623983 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:13.623889    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:13.756223 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:13.756190    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:13.902140 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:13.902112    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"]
Sep 18 18:54:13.904607 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:54:13.904566    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e344933_c927_4f6f_8203_7e3cf2028de5.slice/crio-a6e3389a1b0a2eff199fa2d03d0543d6577141b6c0523e80e1d9e72b66a2c063 WatchSource:0}: Error finding container a6e3389a1b0a2eff199fa2d03d0543d6577141b6c0523e80e1d9e72b66a2c063: Status 404 returned error can't find the container with id a6e3389a1b0a2eff199fa2d03d0543d6577141b6c0523e80e1d9e72b66a2c063
Sep 18 18:54:14.175024 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:14.174942    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6" event={"ID":"6e344933-c927-4f6f-8203-7e3cf2028de5","Type":"ContainerStarted","Data":"a6e3389a1b0a2eff199fa2d03d0543d6577141b6c0523e80e1d9e72b66a2c063"}
Sep 18 18:54:14.734459 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:14.734427    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:15.182071 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:15.182035    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6" event={"ID":"6e344933-c927-4f6f-8203-7e3cf2028de5","Type":"ContainerStarted","Data":"eafa29109e5a349d819399f467954512c497ca4eb9da305f766b42582030df59"}
Sep 18 18:54:15.182262 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:15.182105    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:15.200064 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:15.200018    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6" podStartSLOduration=2.8387372060000002 podStartE2EDuration="3.200002561s" totalImagesPullingTime="361.265355ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:54:15.197978931 +0000 UTC m=+891.396463156" watchObservedRunningTime="2026-09-18 18:54:15.200002561 +0000 UTC m=+891.398486766"
Sep 18 18:54:15.437185 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:15.437106    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:17.747162 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:17.747127    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:19.145770 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:19.145744    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn"
Sep 18 18:54:21.193056 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:21.193029    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:25.796090 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:25.796064    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bwsg_636f0979-fe2e-49fd-86d6-31bd7961e736/ovn-acl-logging/0.log"
Sep 18 18:54:25.810027 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:25.810006    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bwsg_636f0979-fe2e-49fd-86d6-31bd7961e736/ovn-acl-logging/0.log"
Sep 18 18:54:26.933486 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:26.933454    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:28.135689 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:28.135655    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:28.948305 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:28.948273    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:29.611524 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:29.611489    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:30.439616 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:30.439579    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:31.264308 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:31.264275    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:31.944645 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:31.944612    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:32.393282 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:32.393255    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:32.842305 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:32.842273    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:33.526614 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:33.526574    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:34.255462 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:34.255427    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:35.011770 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:35.011734    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:35.654236 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:35.654196    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:41.220364 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:41.220330    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"]
Sep 18 18:54:41.222582 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:41.220528    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6" podUID="6e344933-c927-4f6f-8203-7e3cf2028de5" containerName="maas-api" containerID="cri-o://eafa29109e5a349d819399f467954512c497ca4eb9da305f766b42582030df59" gracePeriod=30
Sep 18 18:54:41.248389 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:41.248319    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:41.484615 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:41.484536    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:41.533863 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:41.533830    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhqtm\" (UniqueName: \"kubernetes.io/projected/6e344933-c927-4f6f-8203-7e3cf2028de5-kube-api-access-fhqtm\") pod \"6e344933-c927-4f6f-8203-7e3cf2028de5\" (UID: \"6e344933-c927-4f6f-8203-7e3cf2028de5\") "
Sep 18 18:54:41.534051 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:41.533882    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e344933-c927-4f6f-8203-7e3cf2028de5-metrics-certs\") pod \"6e344933-c927-4f6f-8203-7e3cf2028de5\" (UID: \"6e344933-c927-4f6f-8203-7e3cf2028de5\") "
Sep 18 18:54:41.534051 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:41.533957    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e344933-c927-4f6f-8203-7e3cf2028de5-openshift-service-ca\") pod \"6e344933-c927-4f6f-8203-7e3cf2028de5\" (UID: \"6e344933-c927-4f6f-8203-7e3cf2028de5\") "
Sep 18 18:54:41.534051 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:41.533992    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e344933-c927-4f6f-8203-7e3cf2028de5-maas-api-tls\") pod \"6e344933-c927-4f6f-8203-7e3cf2028de5\" (UID: \"6e344933-c927-4f6f-8203-7e3cf2028de5\") "
Sep 18 18:54:41.534340 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:41.534313    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e344933-c927-4f6f-8203-7e3cf2028de5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6e344933-c927-4f6f-8203-7e3cf2028de5" (UID: "6e344933-c927-4f6f-8203-7e3cf2028de5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:54:41.536025 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:41.535999    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e344933-c927-4f6f-8203-7e3cf2028de5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6e344933-c927-4f6f-8203-7e3cf2028de5" (UID: "6e344933-c927-4f6f-8203-7e3cf2028de5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:54:41.536122 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:41.536037    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e344933-c927-4f6f-8203-7e3cf2028de5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6e344933-c927-4f6f-8203-7e3cf2028de5" (UID: "6e344933-c927-4f6f-8203-7e3cf2028de5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:54:41.536122 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:41.536056    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e344933-c927-4f6f-8203-7e3cf2028de5-kube-api-access-fhqtm" (OuterVolumeSpecName: "kube-api-access-fhqtm") pod "6e344933-c927-4f6f-8203-7e3cf2028de5" (UID: "6e344933-c927-4f6f-8203-7e3cf2028de5"). InnerVolumeSpecName "kube-api-access-fhqtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:54:41.634963 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:41.634930    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e344933-c927-4f6f-8203-7e3cf2028de5-openshift-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:54:41.634963 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:41.634959    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e344933-c927-4f6f-8203-7e3cf2028de5-maas-api-tls\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:54:41.634963 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:41.634970    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhqtm\" (UniqueName: \"kubernetes.io/projected/6e344933-c927-4f6f-8203-7e3cf2028de5-kube-api-access-fhqtm\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:54:41.635209 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:41.634978    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e344933-c927-4f6f-8203-7e3cf2028de5-metrics-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:54:42.327377 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:42.327312    2558 generic.go:356] "Generic (PLEG): container finished" podID="6e344933-c927-4f6f-8203-7e3cf2028de5" containerID="eafa29109e5a349d819399f467954512c497ca4eb9da305f766b42582030df59" exitCode=0
Sep 18 18:54:42.327879 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:42.327384    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"
Sep 18 18:54:42.327879 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:42.327411    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6" event={"ID":"6e344933-c927-4f6f-8203-7e3cf2028de5","Type":"ContainerDied","Data":"eafa29109e5a349d819399f467954512c497ca4eb9da305f766b42582030df59"}
Sep 18 18:54:42.327879 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:42.327492    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6" event={"ID":"6e344933-c927-4f6f-8203-7e3cf2028de5","Type":"ContainerDied","Data":"a6e3389a1b0a2eff199fa2d03d0543d6577141b6c0523e80e1d9e72b66a2c063"}
Sep 18 18:54:42.327879 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:42.327523    2558 scope.go:117] "RemoveContainer" containerID="eafa29109e5a349d819399f467954512c497ca4eb9da305f766b42582030df59"
Sep 18 18:54:42.336919 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:42.336890    2558 scope.go:117] "RemoveContainer" containerID="eafa29109e5a349d819399f467954512c497ca4eb9da305f766b42582030df59"
Sep 18 18:54:42.337192 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:54:42.337172    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eafa29109e5a349d819399f467954512c497ca4eb9da305f766b42582030df59\": container with ID starting with eafa29109e5a349d819399f467954512c497ca4eb9da305f766b42582030df59 not found: ID does not exist" containerID="eafa29109e5a349d819399f467954512c497ca4eb9da305f766b42582030df59"
Sep 18 18:54:42.337243 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:42.337201    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eafa29109e5a349d819399f467954512c497ca4eb9da305f766b42582030df59"} err="failed to get container status \"eafa29109e5a349d819399f467954512c497ca4eb9da305f766b42582030df59\": rpc error: code = NotFound desc = could not find container \"eafa29109e5a349d819399f467954512c497ca4eb9da305f766b42582030df59\": container with ID starting with eafa29109e5a349d819399f467954512c497ca4eb9da305f766b42582030df59 not found: ID does not exist"
Sep 18 18:54:42.354375 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:42.354341    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"]
Sep 18 18:54:42.363524 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:42.363487    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e8c645ee-cd8bd8b8d-9l6g6"]
Sep 18 18:54:42.541799 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:42.541770    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:42.642912 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:42.642808    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:42.837199 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:42.837164    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:43.514881 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:43.514840    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:43.852727 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:43.852636    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e344933-c927-4f6f-8203-7e3cf2028de5" path="/var/lib/kubelet/pods/6e344933-c927-4f6f-8203-7e3cf2028de5/volumes"
Sep 18 18:54:44.346534 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:44.346498    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:45.111805 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:45.111768    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:45.208066 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:45.208030    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:45.859362 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:45.859332    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:46.158521 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:46.158424    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:46.845466 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:46.845432    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:47.438641 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:47.438610    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:48.238753 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:48.238717    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:48.903956 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:48.903924    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"]
Sep 18 18:54:48.904562 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:48.904539    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e344933-c927-4f6f-8203-7e3cf2028de5" containerName="maas-api"
Sep 18 18:54:48.904708 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:48.904695    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e344933-c927-4f6f-8203-7e3cf2028de5" containerName="maas-api"
Sep 18 18:54:48.904953 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:48.904934    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e344933-c927-4f6f-8203-7e3cf2028de5" containerName="maas-api"
Sep 18 18:54:48.909640 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:48.909623    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:54:48.912047 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:48.912029    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b0eada48-serving-cert\""
Sep 18 18:54:48.919012 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:48.918991    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"]
Sep 18 18:54:48.933982 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:48.933960    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:48.999982 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:48.999956    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae29db99-82b6-428d-a2e5-9d0fe48c3823-maas-api-tls\") pod \"maas-api-e2e-ait-b0eada48-56c77cff5-k94sg\" (UID: \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:54:49.000151 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:49.000009    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae29db99-82b6-428d-a2e5-9d0fe48c3823-openshift-service-ca\") pod \"maas-api-e2e-ait-b0eada48-56c77cff5-k94sg\" (UID: \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:54:49.000151 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:49.000075    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwkz2\" (UniqueName: \"kubernetes.io/projected/ae29db99-82b6-428d-a2e5-9d0fe48c3823-kube-api-access-gwkz2\") pod \"maas-api-e2e-ait-b0eada48-56c77cff5-k94sg\" (UID: \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:54:49.000276 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:49.000215    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae29db99-82b6-428d-a2e5-9d0fe48c3823-metrics-certs\") pod \"maas-api-e2e-ait-b0eada48-56c77cff5-k94sg\" (UID: \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:54:49.035010 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:49.034975    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:49.100985 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:49.100961    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae29db99-82b6-428d-a2e5-9d0fe48c3823-metrics-certs\") pod \"maas-api-e2e-ait-b0eada48-56c77cff5-k94sg\" (UID: \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:54:49.101103 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:49.101007    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae29db99-82b6-428d-a2e5-9d0fe48c3823-maas-api-tls\") pod \"maas-api-e2e-ait-b0eada48-56c77cff5-k94sg\" (UID: \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:54:49.101103 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:49.101043    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae29db99-82b6-428d-a2e5-9d0fe48c3823-openshift-service-ca\") pod \"maas-api-e2e-ait-b0eada48-56c77cff5-k94sg\" (UID: \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:54:49.101103 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:49.101064    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwkz2\" (UniqueName: \"kubernetes.io/projected/ae29db99-82b6-428d-a2e5-9d0fe48c3823-kube-api-access-gwkz2\") pod \"maas-api-e2e-ait-b0eada48-56c77cff5-k94sg\" (UID: \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:54:49.101320 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:54:49.101170    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-serving-cert: secret "maas-api-e2e-ait-b0eada48-serving-cert" not found
Sep 18 18:54:49.101320 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:54:49.101259    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae29db99-82b6-428d-a2e5-9d0fe48c3823-maas-api-tls podName:ae29db99-82b6-428d-a2e5-9d0fe48c3823 nodeName:}" failed. No retries permitted until 2026-09-18 18:54:49.601239221 +0000 UTC m=+925.799723406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ae29db99-82b6-428d-a2e5-9d0fe48c3823-maas-api-tls") pod "maas-api-e2e-ait-b0eada48-56c77cff5-k94sg" (UID: "ae29db99-82b6-428d-a2e5-9d0fe48c3823") : secret "maas-api-e2e-ait-b0eada48-serving-cert" not found
Sep 18 18:54:49.101802 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:49.101783    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae29db99-82b6-428d-a2e5-9d0fe48c3823-openshift-service-ca\") pod \"maas-api-e2e-ait-b0eada48-56c77cff5-k94sg\" (UID: \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:54:49.103557 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:49.103538    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae29db99-82b6-428d-a2e5-9d0fe48c3823-metrics-certs\") pod \"maas-api-e2e-ait-b0eada48-56c77cff5-k94sg\" (UID: \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:54:49.109553 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:49.109535    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwkz2\" (UniqueName: \"kubernetes.io/projected/ae29db99-82b6-428d-a2e5-9d0fe48c3823-kube-api-access-gwkz2\") pod \"maas-api-e2e-ait-b0eada48-56c77cff5-k94sg\" (UID: \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:54:49.294956 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:49.294922    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:49.509562 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:49.509517    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:49.598032 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:49.597953    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:49.605202 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:49.605175    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae29db99-82b6-428d-a2e5-9d0fe48c3823-maas-api-tls\") pod \"maas-api-e2e-ait-b0eada48-56c77cff5-k94sg\" (UID: \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:54:49.607451 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:49.607434    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae29db99-82b6-428d-a2e5-9d0fe48c3823-maas-api-tls\") pod \"maas-api-e2e-ait-b0eada48-56c77cff5-k94sg\" (UID: \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:54:49.822509 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:49.822474    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:54:50.103048 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:50.103017    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"]
Sep 18 18:54:50.104313 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:54:50.104284    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae29db99_82b6_428d_a2e5_9d0fe48c3823.slice/crio-1bd64d31f5a93a36d425237f54144dd40bc79efd433519c8b5ee7bd4b7112161 WatchSource:0}: Error finding container 1bd64d31f5a93a36d425237f54144dd40bc79efd433519c8b5ee7bd4b7112161: Status 404 returned error can't find the container with id 1bd64d31f5a93a36d425237f54144dd40bc79efd433519c8b5ee7bd4b7112161
Sep 18 18:54:50.369516 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:50.369475    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg" event={"ID":"ae29db99-82b6-428d-a2e5-9d0fe48c3823","Type":"ContainerStarted","Data":"1bd64d31f5a93a36d425237f54144dd40bc79efd433519c8b5ee7bd4b7112161"}
Sep 18 18:54:51.376119 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:51.376076    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg" event={"ID":"ae29db99-82b6-428d-a2e5-9d0fe48c3823","Type":"ContainerStarted","Data":"af56c19573a4d908104017c5b41ee4ef7f508a437371a97e05cff405da2d1b5e"}
Sep 18 18:54:51.376494 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:51.376164    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:54:51.395043 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:51.394994    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg" podStartSLOduration=2.883340064 podStartE2EDuration="3.394978118s" totalImagesPullingTime="511.638054ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:54:51.3925676 +0000 UTC m=+927.591051825" watchObservedRunningTime="2026-09-18 18:54:51.394978118 +0000 UTC m=+927.593462322"
Sep 18 18:54:51.434949 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:51.434904    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:54:57.386434 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:54:57.386405    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:55:07.448206 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:07.448165    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:12.148427 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:12.148369    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:12.918550 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:12.918515    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:13.837734 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:13.837695    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:14.720252 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:14.720218    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:15.684180 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:15.684137    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:16.168364 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:16.168326    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:16.535582 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:16.535547    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:17.281364 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:17.281336    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:18.142714 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:18.142684    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:18.951749 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:18.951715    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:19.700467 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:19.700431    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:20.425665 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:20.425629    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:26.642070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:26.642032    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:27.480778 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.480743    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"]
Sep 18 18:55:27.481107 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.481080    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg" podUID="ae29db99-82b6-428d-a2e5-9d0fe48c3823" containerName="maas-api" containerID="cri-o://af56c19573a4d908104017c5b41ee4ef7f508a437371a97e05cff405da2d1b5e" gracePeriod=30
Sep 18 18:55:27.540342 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.540309    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:27.654417 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.654380    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:27.739791 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.739734    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:55:27.771504 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.771473    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwkz2\" (UniqueName: \"kubernetes.io/projected/ae29db99-82b6-428d-a2e5-9d0fe48c3823-kube-api-access-gwkz2\") pod \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\" (UID: \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\") "
Sep 18 18:55:27.771652 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.771522    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae29db99-82b6-428d-a2e5-9d0fe48c3823-maas-api-tls\") pod \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\" (UID: \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\") "
Sep 18 18:55:27.771717 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.771665    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae29db99-82b6-428d-a2e5-9d0fe48c3823-openshift-service-ca\") pod \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\" (UID: \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\") "
Sep 18 18:55:27.771777 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.771768    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae29db99-82b6-428d-a2e5-9d0fe48c3823-metrics-certs\") pod \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\" (UID: \"ae29db99-82b6-428d-a2e5-9d0fe48c3823\") "
Sep 18 18:55:27.772078 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.772049    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae29db99-82b6-428d-a2e5-9d0fe48c3823-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ae29db99-82b6-428d-a2e5-9d0fe48c3823" (UID: "ae29db99-82b6-428d-a2e5-9d0fe48c3823"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:55:27.772216 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.772179    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae29db99-82b6-428d-a2e5-9d0fe48c3823-openshift-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:55:27.773777 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.773748    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae29db99-82b6-428d-a2e5-9d0fe48c3823-kube-api-access-gwkz2" (OuterVolumeSpecName: "kube-api-access-gwkz2") pod "ae29db99-82b6-428d-a2e5-9d0fe48c3823" (UID: "ae29db99-82b6-428d-a2e5-9d0fe48c3823"). InnerVolumeSpecName "kube-api-access-gwkz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:55:27.773911 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.773775    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae29db99-82b6-428d-a2e5-9d0fe48c3823-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ae29db99-82b6-428d-a2e5-9d0fe48c3823" (UID: "ae29db99-82b6-428d-a2e5-9d0fe48c3823"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:55:27.773991 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.773968    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae29db99-82b6-428d-a2e5-9d0fe48c3823-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ae29db99-82b6-428d-a2e5-9d0fe48c3823" (UID: "ae29db99-82b6-428d-a2e5-9d0fe48c3823"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:55:27.816281 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.816243    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:27.873390 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.873358    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae29db99-82b6-428d-a2e5-9d0fe48c3823-metrics-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:55:27.873390 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.873390    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwkz2\" (UniqueName: \"kubernetes.io/projected/ae29db99-82b6-428d-a2e5-9d0fe48c3823-kube-api-access-gwkz2\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:55:27.873538 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.873407    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae29db99-82b6-428d-a2e5-9d0fe48c3823-maas-api-tls\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:55:27.930212 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:27.930166    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:28.022863 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:28.022807    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:28.471185 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:28.471098    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:28.575666 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:28.575627    2558 generic.go:356] "Generic (PLEG): container finished" podID="ae29db99-82b6-428d-a2e5-9d0fe48c3823" containerID="af56c19573a4d908104017c5b41ee4ef7f508a437371a97e05cff405da2d1b5e" exitCode=0
Sep 18 18:55:28.575847 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:28.575676    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"
Sep 18 18:55:28.575847 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:28.575698    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg" event={"ID":"ae29db99-82b6-428d-a2e5-9d0fe48c3823","Type":"ContainerDied","Data":"af56c19573a4d908104017c5b41ee4ef7f508a437371a97e05cff405da2d1b5e"}
Sep 18 18:55:28.575847 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:28.575749    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg" event={"ID":"ae29db99-82b6-428d-a2e5-9d0fe48c3823","Type":"ContainerDied","Data":"1bd64d31f5a93a36d425237f54144dd40bc79efd433519c8b5ee7bd4b7112161"}
Sep 18 18:55:28.575847 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:28.575777    2558 scope.go:117] "RemoveContainer" containerID="af56c19573a4d908104017c5b41ee4ef7f508a437371a97e05cff405da2d1b5e"
Sep 18 18:55:28.584712 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:28.584695    2558 scope.go:117] "RemoveContainer" containerID="af56c19573a4d908104017c5b41ee4ef7f508a437371a97e05cff405da2d1b5e"
Sep 18 18:55:28.584992 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:55:28.584969    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af56c19573a4d908104017c5b41ee4ef7f508a437371a97e05cff405da2d1b5e\": container with ID starting with af56c19573a4d908104017c5b41ee4ef7f508a437371a97e05cff405da2d1b5e not found: ID does not exist" containerID="af56c19573a4d908104017c5b41ee4ef7f508a437371a97e05cff405da2d1b5e"
Sep 18 18:55:28.585090 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:28.584999    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af56c19573a4d908104017c5b41ee4ef7f508a437371a97e05cff405da2d1b5e"} err="failed to get container status \"af56c19573a4d908104017c5b41ee4ef7f508a437371a97e05cff405da2d1b5e\": rpc error: code = NotFound desc = could not find container \"af56c19573a4d908104017c5b41ee4ef7f508a437371a97e05cff405da2d1b5e\": container with ID starting with af56c19573a4d908104017c5b41ee4ef7f508a437371a97e05cff405da2d1b5e not found: ID does not exist"
Sep 18 18:55:28.601111 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:28.601090    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"]
Sep 18 18:55:28.608035 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:28.608016    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0eada48-56c77cff5-k94sg"]
Sep 18 18:55:29.185416 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:29.185352    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:29.307228 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:29.307197    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:29.648653 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:29.648599    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:29.846057 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:29.846025    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae29db99-82b6-428d-a2e5-9d0fe48c3823" path="/var/lib/kubelet/pods/ae29db99-82b6-428d-a2e5-9d0fe48c3823/volumes"
Sep 18 18:55:29.960075 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:29.959982    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:30.380420 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:30.380373    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:30.475585 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:30.475548    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:31.355538 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:31.355493    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:32.050162 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:32.050121    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:32.158926 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:32.158882    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:32.399310 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:32.399183    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:33.442197 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.442164    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"]
Sep 18 18:55:33.442785 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.442765    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae29db99-82b6-428d-a2e5-9d0fe48c3823" containerName="maas-api"
Sep 18 18:55:33.442895 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.442788    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae29db99-82b6-428d-a2e5-9d0fe48c3823" containerName="maas-api"
Sep 18 18:55:33.442956 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.442935    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae29db99-82b6-428d-a2e5-9d0fe48c3823" containerName="maas-api"
Sep 18 18:55:33.447876 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.447853    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:55:33.450523 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.450505    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-d2dcf91d-serving-cert\""
Sep 18 18:55:33.469738 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.469692    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"]
Sep 18 18:55:33.494834 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.494786    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:33.527114 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.527083    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-maas-api-tls\") pod \"maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw\" (UID: \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:55:33.527258 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.527162    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-metrics-certs\") pod \"maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw\" (UID: \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:55:33.527258 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.527222    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-openshift-service-ca\") pod \"maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw\" (UID: \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:55:33.527258 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.527242    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdv5f\" (UniqueName: \"kubernetes.io/projected/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-kube-api-access-gdv5f\") pod \"maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw\" (UID: \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:55:33.611999 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.611968    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:33.627672 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.627654    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdv5f\" (UniqueName: \"kubernetes.io/projected/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-kube-api-access-gdv5f\") pod \"maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw\" (UID: \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:55:33.627767 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.627682    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-maas-api-tls\") pod \"maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw\" (UID: \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:55:33.627767 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.627739    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-metrics-certs\") pod \"maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw\" (UID: \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:55:33.627899 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.627795    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-openshift-service-ca\") pod \"maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw\" (UID: \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:55:33.627955 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:55:33.627903    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-serving-cert: secret "maas-api-e2e-derive-d2dcf91d-serving-cert" not found
Sep 18 18:55:33.628008 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:55:33.627992    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-maas-api-tls podName:a6508f3d-7778-4c5d-ab0e-71dceff2bcd2 nodeName:}" failed. No retries permitted until 2026-09-18 18:55:34.127971058 +0000 UTC m=+970.326455261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-maas-api-tls") pod "maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw" (UID: "a6508f3d-7778-4c5d-ab0e-71dceff2bcd2") : secret "maas-api-e2e-derive-d2dcf91d-serving-cert" not found
Sep 18 18:55:33.628407 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.628389    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-openshift-service-ca\") pod \"maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw\" (UID: \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:55:33.630297 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.630275    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-metrics-certs\") pod \"maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw\" (UID: \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:55:33.641637 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.641614    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdv5f\" (UniqueName: \"kubernetes.io/projected/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-kube-api-access-gdv5f\") pod \"maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw\" (UID: \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:55:33.719371 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:33.719293    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:34.028126 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.028097    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:34.130734 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.130699    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:34.131909 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.131889    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-maas-api-tls\") pod \"maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw\" (UID: \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:55:34.134199 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.134176    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-maas-api-tls\") pod \"maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw\" (UID: \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:55:34.230001 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.229969    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj"]
Sep 18 18:55:34.233788 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.233772    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj"
Sep 18 18:55:34.238319 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.238297    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-d2dcf91d-dockercfg-9l44v\""
Sep 18 18:55:34.238319 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.238315    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-d2dcf91d\""
Sep 18 18:55:34.256767 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.256741    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj"]
Sep 18 18:55:34.333720 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.333648    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmlrd\" (UniqueName: \"kubernetes.io/projected/251501d7-e229-4249-b202-b0ae6aed8e16-kube-api-access-fmlrd\") pod \"payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj\" (UID: \"251501d7-e229-4249-b202-b0ae6aed8e16\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj"
Sep 18 18:55:34.333921 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.333759    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/251501d7-e229-4249-b202-b0ae6aed8e16-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj\" (UID: \"251501d7-e229-4249-b202-b0ae6aed8e16\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj"
Sep 18 18:55:34.358558 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.358533    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:55:34.434489 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.434453    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/251501d7-e229-4249-b202-b0ae6aed8e16-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj\" (UID: \"251501d7-e229-4249-b202-b0ae6aed8e16\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj"
Sep 18 18:55:34.434648 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.434536    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmlrd\" (UniqueName: \"kubernetes.io/projected/251501d7-e229-4249-b202-b0ae6aed8e16-kube-api-access-fmlrd\") pod \"payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj\" (UID: \"251501d7-e229-4249-b202-b0ae6aed8e16\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj"
Sep 18 18:55:34.435436 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.435385    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/251501d7-e229-4249-b202-b0ae6aed8e16-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj\" (UID: \"251501d7-e229-4249-b202-b0ae6aed8e16\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj"
Sep 18 18:55:34.443995 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.443972    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmlrd\" (UniqueName: \"kubernetes.io/projected/251501d7-e229-4249-b202-b0ae6aed8e16-kube-api-access-fmlrd\") pod \"payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj\" (UID: \"251501d7-e229-4249-b202-b0ae6aed8e16\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj"
Sep 18 18:55:34.544335 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.544305    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj"
Sep 18 18:55:34.596134 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.595556    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:34.640429 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.640282    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"]
Sep 18 18:55:34.643566 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:55:34.643542    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6508f3d_7778_4c5d_ab0e_71dceff2bcd2.slice/crio-5614350cd4c4b0d5edc4d517a803fa373dbe49fba650a3187777fb43e340ead0 WatchSource:0}: Error finding container 5614350cd4c4b0d5edc4d517a803fa373dbe49fba650a3187777fb43e340ead0: Status 404 returned error can't find the container with id 5614350cd4c4b0d5edc4d517a803fa373dbe49fba650a3187777fb43e340ead0
Sep 18 18:55:34.645470 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.645453    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:55:34.840961 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:34.840939    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj"]
Sep 18 18:55:34.842924 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:55:34.842863    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod251501d7_e229_4249_b202_b0ae6aed8e16.slice/crio-28df69defebd03ff2f0ef170c59dfeebb3cec7542d83987fb754cbb0edef94d7 WatchSource:0}: Error finding container 28df69defebd03ff2f0ef170c59dfeebb3cec7542d83987fb754cbb0edef94d7: Status 404 returned error can't find the container with id 28df69defebd03ff2f0ef170c59dfeebb3cec7542d83987fb754cbb0edef94d7
Sep 18 18:55:35.480178 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:35.480144    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:35.616316 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:35.616285    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj" event={"ID":"251501d7-e229-4249-b202-b0ae6aed8e16","Type":"ContainerStarted","Data":"40ad8d566e2bc86948ccc5ad918042efb8e4e3aa36f218d6003a345ca31f8407"}
Sep 18 18:55:35.616506 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:35.616338    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj" event={"ID":"251501d7-e229-4249-b202-b0ae6aed8e16","Type":"ContainerStarted","Data":"28df69defebd03ff2f0ef170c59dfeebb3cec7542d83987fb754cbb0edef94d7"}
Sep 18 18:55:35.616506 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:35.616360    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj"
Sep 18 18:55:35.617855 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:35.617829    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw" event={"ID":"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2","Type":"ContainerStarted","Data":"bc77f3edb4413635113d7071b00add68812bf1489e2e33aef5ce70545dbb6dbf"}
Sep 18 18:55:35.617976 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:35.617872    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:55:35.617976 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:35.617892    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw" event={"ID":"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2","Type":"ContainerStarted","Data":"5614350cd4c4b0d5edc4d517a803fa373dbe49fba650a3187777fb43e340ead0"}
Sep 18 18:55:35.635710 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:35.635659    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj" podStartSLOduration=1.635641752 podStartE2EDuration="1.635641752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:55:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:55:35.635149556 +0000 UTC m=+971.833633778" watchObservedRunningTime="2026-09-18 18:55:35.635641752 +0000 UTC m=+971.834125959"
Sep 18 18:55:35.657263 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:35.657222    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw" podStartSLOduration=2.291267166 podStartE2EDuration="2.657209922s" totalImagesPullingTime="365.942756ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:55:35.654304831 +0000 UTC m=+971.852789050" watchObservedRunningTime="2026-09-18 18:55:35.657209922 +0000 UTC m=+971.855694126"
Sep 18 18:55:36.472942 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:36.472911    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:37.194496 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:37.194406    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:38.038513 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:38.038468    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:38.135921 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:38.135886    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:41.630907 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:41.630883    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:55:46.627156 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:46.627125    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj"
Sep 18 18:55:55.599487 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:55.599449    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:56.224568 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:56.224533    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:57.022960 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:57.022918    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:57.967128 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:57.967086    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:58.893899 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:58.893866    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:55:59.640164 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:55:59.640134    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:00.402428 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:00.402386    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:01.148214 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:01.148175    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:02.009598 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:02.009565    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:03.740771 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:03.740737    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:04.316805 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.316764    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"]
Sep 18 18:56:04.317073 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.317041    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw" podUID="a6508f3d-7778-4c5d-ab0e-71dceff2bcd2" containerName="maas-api" containerID="cri-o://bc77f3edb4413635113d7071b00add68812bf1489e2e33aef5ce70545dbb6dbf" gracePeriod=30
Sep 18 18:56:04.346410 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.346379    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:04.386028 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.386000    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj"]
Sep 18 18:56:04.386232 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.386207    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj" podUID="251501d7-e229-4249-b202-b0ae6aed8e16" containerName="payload-pre-processing" containerID="cri-o://40ad8d566e2bc86948ccc5ad918042efb8e4e3aa36f218d6003a345ca31f8407" gracePeriod=30
Sep 18 18:56:04.418184 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.418156    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:04.519941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.519907    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:04.575554 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.575413    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:04.640583 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.640557    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:56:04.674249 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.674226    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj"
Sep 18 18:56:04.731305 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.731279    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/251501d7-e229-4249-b202-b0ae6aed8e16-plugins-config-volume\") pod \"251501d7-e229-4249-b202-b0ae6aed8e16\" (UID: \"251501d7-e229-4249-b202-b0ae6aed8e16\") "
Sep 18 18:56:04.731490 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.731366    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-metrics-certs\") pod \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\" (UID: \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\") "
Sep 18 18:56:04.731490 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.731413    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-openshift-service-ca\") pod \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\" (UID: \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\") "
Sep 18 18:56:04.731490 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.731455    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdv5f\" (UniqueName: \"kubernetes.io/projected/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-kube-api-access-gdv5f\") pod \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\" (UID: \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\") "
Sep 18 18:56:04.732371 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.731572    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-maas-api-tls\") pod \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\" (UID: \"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2\") "
Sep 18 18:56:04.732371 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.731623    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmlrd\" (UniqueName: \"kubernetes.io/projected/251501d7-e229-4249-b202-b0ae6aed8e16-kube-api-access-fmlrd\") pod \"251501d7-e229-4249-b202-b0ae6aed8e16\" (UID: \"251501d7-e229-4249-b202-b0ae6aed8e16\") "
Sep 18 18:56:04.732371 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.731799    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/251501d7-e229-4249-b202-b0ae6aed8e16-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "251501d7-e229-4249-b202-b0ae6aed8e16" (UID: "251501d7-e229-4249-b202-b0ae6aed8e16"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:56:04.732371 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.731918    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a6508f3d-7778-4c5d-ab0e-71dceff2bcd2" (UID: "a6508f3d-7778-4c5d-ab0e-71dceff2bcd2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:56:04.732371 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.732097    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-openshift-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:56:04.732371 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.732119    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/251501d7-e229-4249-b202-b0ae6aed8e16-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:56:04.732707 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.732673    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:04.733837 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.733791    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a6508f3d-7778-4c5d-ab0e-71dceff2bcd2" (UID: "a6508f3d-7778-4c5d-ab0e-71dceff2bcd2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:56:04.734081 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.734050    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a6508f3d-7778-4c5d-ab0e-71dceff2bcd2" (UID: "a6508f3d-7778-4c5d-ab0e-71dceff2bcd2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:56:04.734219 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.734202    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/251501d7-e229-4249-b202-b0ae6aed8e16-kube-api-access-fmlrd" (OuterVolumeSpecName: "kube-api-access-fmlrd") pod "251501d7-e229-4249-b202-b0ae6aed8e16" (UID: "251501d7-e229-4249-b202-b0ae6aed8e16"). InnerVolumeSpecName "kube-api-access-fmlrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:56:04.734272 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.734229    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-kube-api-access-gdv5f" (OuterVolumeSpecName: "kube-api-access-gdv5f") pod "a6508f3d-7778-4c5d-ab0e-71dceff2bcd2" (UID: "a6508f3d-7778-4c5d-ab0e-71dceff2bcd2"). InnerVolumeSpecName "kube-api-access-gdv5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:56:04.771928 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.771885    2558 generic.go:356] "Generic (PLEG): container finished" podID="251501d7-e229-4249-b202-b0ae6aed8e16" containerID="40ad8d566e2bc86948ccc5ad918042efb8e4e3aa36f218d6003a345ca31f8407" exitCode=0
Sep 18 18:56:04.772334 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.771935    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj"
Sep 18 18:56:04.772334 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.771986    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj" event={"ID":"251501d7-e229-4249-b202-b0ae6aed8e16","Type":"ContainerDied","Data":"40ad8d566e2bc86948ccc5ad918042efb8e4e3aa36f218d6003a345ca31f8407"}
Sep 18 18:56:04.772334 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.772062    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj" event={"ID":"251501d7-e229-4249-b202-b0ae6aed8e16","Type":"ContainerDied","Data":"28df69defebd03ff2f0ef170c59dfeebb3cec7542d83987fb754cbb0edef94d7"}
Sep 18 18:56:04.772334 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.772096    2558 scope.go:117] "RemoveContainer" containerID="40ad8d566e2bc86948ccc5ad918042efb8e4e3aa36f218d6003a345ca31f8407"
Sep 18 18:56:04.773911 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.773886    2558 generic.go:356] "Generic (PLEG): container finished" podID="a6508f3d-7778-4c5d-ab0e-71dceff2bcd2" containerID="bc77f3edb4413635113d7071b00add68812bf1489e2e33aef5ce70545dbb6dbf" exitCode=0
Sep 18 18:56:04.774034 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.773941    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"
Sep 18 18:56:04.774034 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.773942    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw" event={"ID":"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2","Type":"ContainerDied","Data":"bc77f3edb4413635113d7071b00add68812bf1489e2e33aef5ce70545dbb6dbf"}
Sep 18 18:56:04.774149 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.774070    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw" event={"ID":"a6508f3d-7778-4c5d-ab0e-71dceff2bcd2","Type":"ContainerDied","Data":"5614350cd4c4b0d5edc4d517a803fa373dbe49fba650a3187777fb43e340ead0"}
Sep 18 18:56:04.785444 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.785409    2558 scope.go:117] "RemoveContainer" containerID="40ad8d566e2bc86948ccc5ad918042efb8e4e3aa36f218d6003a345ca31f8407"
Sep 18 18:56:04.785759 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:56:04.785740    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40ad8d566e2bc86948ccc5ad918042efb8e4e3aa36f218d6003a345ca31f8407\": container with ID starting with 40ad8d566e2bc86948ccc5ad918042efb8e4e3aa36f218d6003a345ca31f8407 not found: ID does not exist" containerID="40ad8d566e2bc86948ccc5ad918042efb8e4e3aa36f218d6003a345ca31f8407"
Sep 18 18:56:04.785927 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.785774    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40ad8d566e2bc86948ccc5ad918042efb8e4e3aa36f218d6003a345ca31f8407"} err="failed to get container status \"40ad8d566e2bc86948ccc5ad918042efb8e4e3aa36f218d6003a345ca31f8407\": rpc error: code = NotFound desc = could not find container \"40ad8d566e2bc86948ccc5ad918042efb8e4e3aa36f218d6003a345ca31f8407\": container with ID starting with 40ad8d566e2bc86948ccc5ad918042efb8e4e3aa36f218d6003a345ca31f8407 not found: ID does not exist"
Sep 18 18:56:04.785927 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.785801    2558 scope.go:117] "RemoveContainer" containerID="bc77f3edb4413635113d7071b00add68812bf1489e2e33aef5ce70545dbb6dbf"
Sep 18 18:56:04.795012 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.794991    2558 scope.go:117] "RemoveContainer" containerID="bc77f3edb4413635113d7071b00add68812bf1489e2e33aef5ce70545dbb6dbf"
Sep 18 18:56:04.795289 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:56:04.795270    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc77f3edb4413635113d7071b00add68812bf1489e2e33aef5ce70545dbb6dbf\": container with ID starting with bc77f3edb4413635113d7071b00add68812bf1489e2e33aef5ce70545dbb6dbf not found: ID does not exist" containerID="bc77f3edb4413635113d7071b00add68812bf1489e2e33aef5ce70545dbb6dbf"
Sep 18 18:56:04.795358 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.795301    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc77f3edb4413635113d7071b00add68812bf1489e2e33aef5ce70545dbb6dbf"} err="failed to get container status \"bc77f3edb4413635113d7071b00add68812bf1489e2e33aef5ce70545dbb6dbf\": rpc error: code = NotFound desc = could not find container \"bc77f3edb4413635113d7071b00add68812bf1489e2e33aef5ce70545dbb6dbf\": container with ID starting with bc77f3edb4413635113d7071b00add68812bf1489e2e33aef5ce70545dbb6dbf not found: ID does not exist"
Sep 18 18:56:04.802460 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.802437    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"]
Sep 18 18:56:04.804399 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.804381    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d2dcf91d-7cb6d7cf8d-ks4qw"]
Sep 18 18:56:04.816977 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.816959    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj"]
Sep 18 18:56:04.822630 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.822612    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d2dcf91d-76974566f8-qxszj"]
Sep 18 18:56:04.832924 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.832875    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-metrics-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:56:04.832924 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.832894    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdv5f\" (UniqueName: \"kubernetes.io/projected/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-kube-api-access-gdv5f\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:56:04.832924 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.832905    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2-maas-api-tls\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:56:04.832924 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:04.832914    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmlrd\" (UniqueName: \"kubernetes.io/projected/251501d7-e229-4249-b202-b0ae6aed8e16-kube-api-access-fmlrd\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:56:05.035397 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:05.035358    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:05.435347 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:05.435309    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:05.533156 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:05.533124    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:05.632979 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:05.632876    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:05.756120 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:05.756084    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:05.845785 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:05.845744    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="251501d7-e229-4249-b202-b0ae6aed8e16" path="/var/lib/kubelet/pods/251501d7-e229-4249-b202-b0ae6aed8e16/volumes"
Sep 18 18:56:05.846346 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:05.846325    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6508f3d-7778-4c5d-ab0e-71dceff2bcd2" path="/var/lib/kubelet/pods/a6508f3d-7778-4c5d-ab0e-71dceff2bcd2/volumes"
Sep 18 18:56:06.642893 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:06.642858    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:06.977248 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:06.977174    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:07.390069 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:07.390039    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:08.075376 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:08.075348    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:12.759373 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:12.759337    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:12.872461 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:12.872426    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:12.978224 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:12.978197    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:13.145246 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:13.145167    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:13.525456 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:13.525427    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:28.349022 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.348988    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"]
Sep 18 18:56:28.349660 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.349641    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6508f3d-7778-4c5d-ab0e-71dceff2bcd2" containerName="maas-api"
Sep 18 18:56:28.349660 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.349661    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6508f3d-7778-4c5d-ab0e-71dceff2bcd2" containerName="maas-api"
Sep 18 18:56:28.349773 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.349678    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="251501d7-e229-4249-b202-b0ae6aed8e16" containerName="payload-pre-processing"
Sep 18 18:56:28.349773 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.349683    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="251501d7-e229-4249-b202-b0ae6aed8e16" containerName="payload-pre-processing"
Sep 18 18:56:28.349773 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.349753    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="251501d7-e229-4249-b202-b0ae6aed8e16" containerName="payload-pre-processing"
Sep 18 18:56:28.349773 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.349770    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6508f3d-7778-4c5d-ab0e-71dceff2bcd2" containerName="maas-api"
Sep 18 18:56:28.354027 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.354012    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.356536 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.356514    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-897f61\"/\"openshift-service-ca.crt\""
Sep 18 18:56:28.357634 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.357540    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-897f61\"/\"test-model-897f61-kserve-self-signed-certs\""
Sep 18 18:56:28.358211 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.358190    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-897f61\"/\"default-dockercfg-9gkxm\""
Sep 18 18:56:28.358626 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.358588    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-897f61\"/\"kube-root-ca.crt\""
Sep 18 18:56:28.361287 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.361268    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"]
Sep 18 18:56:28.377314 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.377284    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:28.464520 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.464495    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3368d472-a164-4110-bcc3-1f6aa518f526-tls-certs\") pod \"test-model-897f61-kserve-78457fc9d7-5qn5l\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") " pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.464671 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.464547    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqdgc\" (UniqueName: \"kubernetes.io/projected/3368d472-a164-4110-bcc3-1f6aa518f526-kube-api-access-pqdgc\") pod \"test-model-897f61-kserve-78457fc9d7-5qn5l\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") " pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.464671 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.464586    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-home\") pod \"test-model-897f61-kserve-78457fc9d7-5qn5l\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") " pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.464671 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.464655    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-model-cache\") pod \"test-model-897f61-kserve-78457fc9d7-5qn5l\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") " pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.464804 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.464683    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-dshm\") pod \"test-model-897f61-kserve-78457fc9d7-5qn5l\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") " pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.464804 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.464744    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-tmp-dir\") pod \"test-model-897f61-kserve-78457fc9d7-5qn5l\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") " pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.565465 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.565440    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-model-cache\") pod \"test-model-897f61-kserve-78457fc9d7-5qn5l\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") " pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.565640 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.565471    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-dshm\") pod \"test-model-897f61-kserve-78457fc9d7-5qn5l\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") " pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.565640 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.565496    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-tmp-dir\") pod \"test-model-897f61-kserve-78457fc9d7-5qn5l\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") " pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.565640 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.565552    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3368d472-a164-4110-bcc3-1f6aa518f526-tls-certs\") pod \"test-model-897f61-kserve-78457fc9d7-5qn5l\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") " pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.565640 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.565579    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqdgc\" (UniqueName: \"kubernetes.io/projected/3368d472-a164-4110-bcc3-1f6aa518f526-kube-api-access-pqdgc\") pod \"test-model-897f61-kserve-78457fc9d7-5qn5l\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") " pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.565640 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.565604    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-home\") pod \"test-model-897f61-kserve-78457fc9d7-5qn5l\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") " pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.565950 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.565884    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-model-cache\") pod \"test-model-897f61-kserve-78457fc9d7-5qn5l\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") " pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.565950 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.565941    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-tmp-dir\") pod \"test-model-897f61-kserve-78457fc9d7-5qn5l\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") " pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.566070 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.566000    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-home\") pod \"test-model-897f61-kserve-78457fc9d7-5qn5l\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") " pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.567701 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.567680    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-dshm\") pod \"test-model-897f61-kserve-78457fc9d7-5qn5l\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") " pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.568038 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.568021    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3368d472-a164-4110-bcc3-1f6aa518f526-tls-certs\") pod \"test-model-897f61-kserve-78457fc9d7-5qn5l\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") " pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.574043 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.574024    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqdgc\" (UniqueName: \"kubernetes.io/projected/3368d472-a164-4110-bcc3-1f6aa518f526-kube-api-access-pqdgc\") pod \"test-model-897f61-kserve-78457fc9d7-5qn5l\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") " pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.665758 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.665677    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:28.937565 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:28.937543    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"]
Sep 18 18:56:29.902398 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:29.902364    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l" event={"ID":"3368d472-a164-4110-bcc3-1f6aa518f526","Type":"ContainerStarted","Data":"11fdfedf447c76e2b5ae9812df051878c20583d7263e309f223592ecfc49cd0a"}
Sep 18 18:56:29.902398 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:29.902399    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l" event={"ID":"3368d472-a164-4110-bcc3-1f6aa518f526","Type":"ContainerStarted","Data":"aed256bc23088f13448699e3a82cabcb8c0d635ed0757b10f95903d1ba54880e"}
Sep 18 18:56:29.942679 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:29.942632    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l" podStartSLOduration=1.942616491 podStartE2EDuration="1.942616491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:56:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:56:29.942316645 +0000 UTC m=+1026.140800849" watchObservedRunningTime="2026-09-18 18:56:29.942616491 +0000 UTC m=+1026.141100749"
Sep 18 18:56:33.044155 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.044120    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"]
Sep 18 18:56:33.048131 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.048107    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.053764 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.053487    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-9d9f36\"/\"kube-root-ca.crt\""
Sep 18 18:56:33.053764 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.053495    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9d9f36\"/\"distinct-w3-9d9f36-kserve-self-signed-certs\""
Sep 18 18:56:33.053764 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.053497    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-9d9f36\"/\"openshift-service-ca.crt\""
Sep 18 18:56:33.053764 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.053495    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9d9f36\"/\"default-dockercfg-2khll\""
Sep 18 18:56:33.062959 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.062937    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"]
Sep 18 18:56:33.087154 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.087117    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:33.107228 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.107202    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-dshm\") pod \"distinct-w3-9d9f36-kserve-8645cf648c-6frhr\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.107383 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.107244    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-home\") pod \"distinct-w3-9d9f36-kserve-8645cf648c-6frhr\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.107383 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.107272    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09416ddf-4f59-44aa-88e8-61399586938a-tls-certs\") pod \"distinct-w3-9d9f36-kserve-8645cf648c-6frhr\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.107474 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.107402    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-tmp-dir\") pod \"distinct-w3-9d9f36-kserve-8645cf648c-6frhr\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.107474 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.107445    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-model-cache\") pod \"distinct-w3-9d9f36-kserve-8645cf648c-6frhr\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.107548 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.107518    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q862q\" (UniqueName: \"kubernetes.io/projected/09416ddf-4f59-44aa-88e8-61399586938a-kube-api-access-q862q\") pod \"distinct-w3-9d9f36-kserve-8645cf648c-6frhr\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.208623 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.208588    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q862q\" (UniqueName: \"kubernetes.io/projected/09416ddf-4f59-44aa-88e8-61399586938a-kube-api-access-q862q\") pod \"distinct-w3-9d9f36-kserve-8645cf648c-6frhr\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.208794 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.208642    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-dshm\") pod \"distinct-w3-9d9f36-kserve-8645cf648c-6frhr\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.208794 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.208674    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-home\") pod \"distinct-w3-9d9f36-kserve-8645cf648c-6frhr\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.208960 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.208859    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09416ddf-4f59-44aa-88e8-61399586938a-tls-certs\") pod \"distinct-w3-9d9f36-kserve-8645cf648c-6frhr\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.208960 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.208951    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-tmp-dir\") pod \"distinct-w3-9d9f36-kserve-8645cf648c-6frhr\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.209073 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.208977    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-model-cache\") pod \"distinct-w3-9d9f36-kserve-8645cf648c-6frhr\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.209115 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.209079    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-home\") pod \"distinct-w3-9d9f36-kserve-8645cf648c-6frhr\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.209303 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.209280    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-tmp-dir\") pod \"distinct-w3-9d9f36-kserve-8645cf648c-6frhr\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.209377 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.209316    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-model-cache\") pod \"distinct-w3-9d9f36-kserve-8645cf648c-6frhr\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.210928 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.210898    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-dshm\") pod \"distinct-w3-9d9f36-kserve-8645cf648c-6frhr\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.211192 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.211174    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09416ddf-4f59-44aa-88e8-61399586938a-tls-certs\") pod \"distinct-w3-9d9f36-kserve-8645cf648c-6frhr\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.216136 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.216120    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q862q\" (UniqueName: \"kubernetes.io/projected/09416ddf-4f59-44aa-88e8-61399586938a-kube-api-access-q862q\") pod \"distinct-w3-9d9f36-kserve-8645cf648c-6frhr\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") " pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.358330 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.358276    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:33.636680 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.636537    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"]
Sep 18 18:56:33.639272 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:56:33.639243    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09416ddf_4f59_44aa_88e8_61399586938a.slice/crio-652f659957e94e677f200977496e8773c3076c89335ecedb35d6b1974dcda59c WatchSource:0}: Error finding container 652f659957e94e677f200977496e8773c3076c89335ecedb35d6b1974dcda59c: Status 404 returned error can't find the container with id 652f659957e94e677f200977496e8773c3076c89335ecedb35d6b1974dcda59c
Sep 18 18:56:33.923796 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.923708    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr" event={"ID":"09416ddf-4f59-44aa-88e8-61399586938a","Type":"ContainerStarted","Data":"536073195b3c78371a5d8347e00d0c8b745fdda6a76715f7b2c6c0e75a21a6d9"}
Sep 18 18:56:33.923796 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.923760    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr" event={"ID":"09416ddf-4f59-44aa-88e8-61399586938a","Type":"ContainerStarted","Data":"652f659957e94e677f200977496e8773c3076c89335ecedb35d6b1974dcda59c"}
Sep 18 18:56:33.942095 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:33.942051    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr" podStartSLOduration=0.942038575 podStartE2EDuration="942.038575ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:56:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:56:33.94009014 +0000 UTC m=+1030.138574375" watchObservedRunningTime="2026-09-18 18:56:33.942038575 +0000 UTC m=+1030.140522778"
Sep 18 18:56:38.666722 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:38.666688    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:38.666722 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:38.666733    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:38.679332 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:38.679308    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:38.962688 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:38.962613    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:56:39.491246 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:39.491196    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:43.165454 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:43.165423    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:43.358376 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:43.358349    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:43.358545 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:43.358389    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:43.371776 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:43.371753    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:43.992343 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:43.992317    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 18:56:45.772426 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:45.772394    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:45.883813 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:45.883777    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:45.975017 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:45.974988    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:56:51.391528 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:56:51.391486    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:01.742146 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:01.742101    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:08.040584 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:08.040493    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:08.147227 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:08.147185    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:08.241598 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:08.241552    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:13.978582 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:13.978545    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:18.693956 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:18.693911    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:19.434646 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:19.434610    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:20.756088 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:20.756052    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:25.040926 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:25.040890    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:25.146357 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:25.146324    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:25.229353 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:25.229318    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:25.285155 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:25.285105    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:25.923320 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:25.923285    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:26.019127 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:26.019080    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:26.118555 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:26.118524    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:27.503332 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:27.503294    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:33.713809 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:33.713778    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:33.787692 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:33.787654    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:34.372272 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:34.372239    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:40.290458 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:40.290398    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:40.354917 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:40.354885    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:40.533271 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:40.533235    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:40.836045 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:40.836009    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:41.374300 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:41.374225    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:43.142971 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:43.142933    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:43.826091 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:43.826058    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:43.927244 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:43.927210    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:44.032027 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:44.031995    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:44.645923 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:44.645887    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:45.422207 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:45.422178    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:46.036813 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.036779    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:46.652317 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.652280    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"]
Sep 18 18:57:46.656751 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.656731    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:57:46.658992 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.658971    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-62de055a-serving-cert\""
Sep 18 18:57:46.663324 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.663300    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"]
Sep 18 18:57:46.681831 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.681789    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:46.741219 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.734743    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g75n5\" (UniqueName: \"kubernetes.io/projected/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-kube-api-access-g75n5\") pod \"maas-api-e2e-mt-62de055a-7fd7c67969-dntw8\" (UID: \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:57:46.741219 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.734939    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-metrics-certs\") pod \"maas-api-e2e-mt-62de055a-7fd7c67969-dntw8\" (UID: \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:57:46.741219 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.734988    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-openshift-service-ca\") pod \"maas-api-e2e-mt-62de055a-7fd7c67969-dntw8\" (UID: \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:57:46.741219 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.735050    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-maas-api-tls\") pod \"maas-api-e2e-mt-62de055a-7fd7c67969-dntw8\" (UID: \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:57:46.784629 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.784595    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:46.836088 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.836040    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-metrics-certs\") pod \"maas-api-e2e-mt-62de055a-7fd7c67969-dntw8\" (UID: \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:57:46.836088 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.836094    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-openshift-service-ca\") pod \"maas-api-e2e-mt-62de055a-7fd7c67969-dntw8\" (UID: \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:57:46.836346 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.836132    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-maas-api-tls\") pod \"maas-api-e2e-mt-62de055a-7fd7c67969-dntw8\" (UID: \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:57:46.836346 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.836176    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g75n5\" (UniqueName: \"kubernetes.io/projected/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-kube-api-access-g75n5\") pod \"maas-api-e2e-mt-62de055a-7fd7c67969-dntw8\" (UID: \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:57:46.836346 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:57:46.836275    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-serving-cert: secret "maas-api-e2e-mt-62de055a-serving-cert" not found
Sep 18 18:57:46.836346 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:57:46.836340    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-maas-api-tls podName:74f726ce-8d89-4a20-865e-1c6e68fa9a7c nodeName:}" failed. No retries permitted until 2026-09-18 18:57:47.336323143 +0000 UTC m=+1103.534807330 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-maas-api-tls") pod "maas-api-e2e-mt-62de055a-7fd7c67969-dntw8" (UID: "74f726ce-8d89-4a20-865e-1c6e68fa9a7c") : secret "maas-api-e2e-mt-62de055a-serving-cert" not found
Sep 18 18:57:46.836881 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.836860    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-openshift-service-ca\") pod \"maas-api-e2e-mt-62de055a-7fd7c67969-dntw8\" (UID: \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:57:46.838596 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.838580    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-metrics-certs\") pod \"maas-api-e2e-mt-62de055a-7fd7c67969-dntw8\" (UID: \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:57:46.844409 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.844384    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g75n5\" (UniqueName: \"kubernetes.io/projected/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-kube-api-access-g75n5\") pod \"maas-api-e2e-mt-62de055a-7fd7c67969-dntw8\" (UID: \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:57:46.937949 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:46.937856    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:47.136793 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:47.136762    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:47.237595 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:47.237499    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:47.342282 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:47.342246    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-maas-api-tls\") pod \"maas-api-e2e-mt-62de055a-7fd7c67969-dntw8\" (UID: \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:57:47.344751 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:47.344728    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-maas-api-tls\") pod \"maas-api-e2e-mt-62de055a-7fd7c67969-dntw8\" (UID: \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:57:47.568335 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:47.568293    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:57:47.680856 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:47.680799    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:47.847418 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:47.847370    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"]
Sep 18 18:57:47.849284 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:57:47.849260    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74f726ce_8d89_4a20_865e_1c6e68fa9a7c.slice/crio-2be3b7205be799e0b7b742ec215a99ac30bd7fba31718964c8842b0d612c8aab WatchSource:0}: Error finding container 2be3b7205be799e0b7b742ec215a99ac30bd7fba31718964c8842b0d612c8aab: Status 404 returned error can't find the container with id 2be3b7205be799e0b7b742ec215a99ac30bd7fba31718964c8842b0d612c8aab
Sep 18 18:57:48.325879 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:48.325846    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8" event={"ID":"74f726ce-8d89-4a20-865e-1c6e68fa9a7c","Type":"ContainerStarted","Data":"2be3b7205be799e0b7b742ec215a99ac30bd7fba31718964c8842b0d612c8aab"}
Sep 18 18:57:48.362377 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:48.362340    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:57:49.333279 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:49.333243    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8" event={"ID":"74f726ce-8d89-4a20-865e-1c6e68fa9a7c","Type":"ContainerStarted","Data":"5f7b0418e6b618e1c129209bcd62f67d1645d9666076109b81f422c903cc86ab"}
Sep 18 18:57:49.333665 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:49.333377    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:57:49.353554 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:49.353510    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8" podStartSLOduration=2.822338629 podStartE2EDuration="3.353497326s" totalImagesPullingTime="531.158697ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:57:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:57:49.351663298 +0000 UTC m=+1105.550147530" watchObservedRunningTime="2026-09-18 18:57:49.353497326 +0000 UTC m=+1105.551981530"
Sep 18 18:57:55.345084 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:57:55.345052    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:58:03.330930 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:03.330895    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:05.132760 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:05.132727    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"]
Sep 18 18:58:05.151091 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:05.151052    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:05.516317 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:05.516277    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:06.172726 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:06.172681    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:06.894390 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:06.894354    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:07.270390 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:07.270354    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:07.567725 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:07.567645    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:08.344434 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:08.344400    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:09.127092 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:09.127047    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:09.323087 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:09.323048    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:09.423840 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:09.423733    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:09.926960 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:09.926928    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:10.726284 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:10.726222    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:11.453501 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:11.453472    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:12.240440 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:12.240408    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:12.930001 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:12.929955    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:13.619673 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:13.619639    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:13.884439 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:13.884348    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:14.542675 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:14.542638    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:14.647333 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:14.647305    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:14.937900 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:14.937800    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:15.410517 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:15.410475    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:15.965778 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:15.965743    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"]
Sep 18 18:58:15.966249 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:15.965996    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49" podUID="850a7519-641a-4a10-9509-6bb7f462b0af" containerName="maas-api" containerID="cri-o://01faf3d6d12bf2ab55397f17344ed1bcc85800ec6dd365ad66a96d6fc97cbb4b" gracePeriod=30
Sep 18 18:58:16.008181 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.008131    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:16.073572 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.073530    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:16.133573 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.133538    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:16.231690 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.231637    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"
Sep 18 18:58:16.237326 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.237300    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:16.311540 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.311504    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:16.335454 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.335421    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnlhz\" (UniqueName: \"kubernetes.io/projected/850a7519-641a-4a10-9509-6bb7f462b0af-kube-api-access-cnlhz\") pod \"850a7519-641a-4a10-9509-6bb7f462b0af\" (UID: \"850a7519-641a-4a10-9509-6bb7f462b0af\") "
Sep 18 18:58:16.335616 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.335545    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/850a7519-641a-4a10-9509-6bb7f462b0af-metrics-certs\") pod \"850a7519-641a-4a10-9509-6bb7f462b0af\" (UID: \"850a7519-641a-4a10-9509-6bb7f462b0af\") "
Sep 18 18:58:16.335616 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.335588    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/850a7519-641a-4a10-9509-6bb7f462b0af-openshift-service-ca\") pod \"850a7519-641a-4a10-9509-6bb7f462b0af\" (UID: \"850a7519-641a-4a10-9509-6bb7f462b0af\") "
Sep 18 18:58:16.335811 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.335625    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/850a7519-641a-4a10-9509-6bb7f462b0af-maas-api-tls\") pod \"850a7519-641a-4a10-9509-6bb7f462b0af\" (UID: \"850a7519-641a-4a10-9509-6bb7f462b0af\") "
Sep 18 18:58:16.336204 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.336162    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/850a7519-641a-4a10-9509-6bb7f462b0af-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "850a7519-641a-4a10-9509-6bb7f462b0af" (UID: "850a7519-641a-4a10-9509-6bb7f462b0af"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:58:16.338158 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.338133    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/850a7519-641a-4a10-9509-6bb7f462b0af-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "850a7519-641a-4a10-9509-6bb7f462b0af" (UID: "850a7519-641a-4a10-9509-6bb7f462b0af"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:58:16.338158 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.338135    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/850a7519-641a-4a10-9509-6bb7f462b0af-kube-api-access-cnlhz" (OuterVolumeSpecName: "kube-api-access-cnlhz") pod "850a7519-641a-4a10-9509-6bb7f462b0af" (UID: "850a7519-641a-4a10-9509-6bb7f462b0af"). InnerVolumeSpecName "kube-api-access-cnlhz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:58:16.338391 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.338371    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/850a7519-641a-4a10-9509-6bb7f462b0af-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "850a7519-641a-4a10-9509-6bb7f462b0af" (UID: "850a7519-641a-4a10-9509-6bb7f462b0af"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:58:16.437088 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.437059    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnlhz\" (UniqueName: \"kubernetes.io/projected/850a7519-641a-4a10-9509-6bb7f462b0af-kube-api-access-cnlhz\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:58:16.437088 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.437083    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/850a7519-641a-4a10-9509-6bb7f462b0af-metrics-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:58:16.437088 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.437093    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/850a7519-641a-4a10-9509-6bb7f462b0af-openshift-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:58:16.437291 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.437101    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/850a7519-641a-4a10-9509-6bb7f462b0af-maas-api-tls\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:58:16.487553 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.487470    2558 generic.go:356] "Generic (PLEG): container finished" podID="850a7519-641a-4a10-9509-6bb7f462b0af" containerID="01faf3d6d12bf2ab55397f17344ed1bcc85800ec6dd365ad66a96d6fc97cbb4b" exitCode=0
Sep 18 18:58:16.487553 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.487518    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"
Sep 18 18:58:16.487741 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.487554    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49" event={"ID":"850a7519-641a-4a10-9509-6bb7f462b0af","Type":"ContainerDied","Data":"01faf3d6d12bf2ab55397f17344ed1bcc85800ec6dd365ad66a96d6fc97cbb4b"}
Sep 18 18:58:16.487741 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.487603    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49" event={"ID":"850a7519-641a-4a10-9509-6bb7f462b0af","Type":"ContainerDied","Data":"764ec44b48133aea40002ce3dd871569ea767581ed030d7603b9b4eaf0ee4e41"}
Sep 18 18:58:16.487741 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.487625    2558 scope.go:117] "RemoveContainer" containerID="01faf3d6d12bf2ab55397f17344ed1bcc85800ec6dd365ad66a96d6fc97cbb4b"
Sep 18 18:58:16.497018 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.497000    2558 scope.go:117] "RemoveContainer" containerID="01faf3d6d12bf2ab55397f17344ed1bcc85800ec6dd365ad66a96d6fc97cbb4b"
Sep 18 18:58:16.497279 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:58:16.497262    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01faf3d6d12bf2ab55397f17344ed1bcc85800ec6dd365ad66a96d6fc97cbb4b\": container with ID starting with 01faf3d6d12bf2ab55397f17344ed1bcc85800ec6dd365ad66a96d6fc97cbb4b not found: ID does not exist" containerID="01faf3d6d12bf2ab55397f17344ed1bcc85800ec6dd365ad66a96d6fc97cbb4b"
Sep 18 18:58:16.497324 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.497287    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01faf3d6d12bf2ab55397f17344ed1bcc85800ec6dd365ad66a96d6fc97cbb4b"} err="failed to get container status \"01faf3d6d12bf2ab55397f17344ed1bcc85800ec6dd365ad66a96d6fc97cbb4b\": rpc error: code = NotFound desc = could not find container \"01faf3d6d12bf2ab55397f17344ed1bcc85800ec6dd365ad66a96d6fc97cbb4b\": container with ID starting with 01faf3d6d12bf2ab55397f17344ed1bcc85800ec6dd365ad66a96d6fc97cbb4b not found: ID does not exist"
Sep 18 18:58:16.509914 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.509893    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"]
Sep 18 18:58:16.514020 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.514001    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bcb770-5546cd67cb-tkl49"]
Sep 18 18:58:16.583828 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.583784    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:16.684293 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:16.684259    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:17.326176 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:17.326141    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:17.844852 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:17.844802    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="850a7519-641a-4a10-9509-6bb7f462b0af" path="/var/lib/kubelet/pods/850a7519-641a-4a10-9509-6bb7f462b0af/volumes"
Sep 18 18:58:18.033076 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:18.033045    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:18.738635 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:18.738600    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:19.300869 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:19.300809    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:20.152843 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.152794    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l" podUID="3368d472-a164-4110-bcc3-1f6aa518f526" containerName="main" containerID="cri-o://11fdfedf447c76e2b5ae9812df051878c20583d7263e309f223592ecfc49cd0a" gracePeriod=45
Sep 18 18:58:20.412469 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.412414    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:58:20.473262 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.473237    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3368d472-a164-4110-bcc3-1f6aa518f526-tls-certs\") pod \"3368d472-a164-4110-bcc3-1f6aa518f526\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") "
Sep 18 18:58:20.473438 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.473283    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-dshm\") pod \"3368d472-a164-4110-bcc3-1f6aa518f526\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") "
Sep 18 18:58:20.473438 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.473313    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqdgc\" (UniqueName: \"kubernetes.io/projected/3368d472-a164-4110-bcc3-1f6aa518f526-kube-api-access-pqdgc\") pod \"3368d472-a164-4110-bcc3-1f6aa518f526\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") "
Sep 18 18:58:20.473438 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.473352    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-home\") pod \"3368d472-a164-4110-bcc3-1f6aa518f526\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") "
Sep 18 18:58:20.473438 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.473407    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-tmp-dir\") pod \"3368d472-a164-4110-bcc3-1f6aa518f526\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") "
Sep 18 18:58:20.473674 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.473447    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-model-cache\") pod \"3368d472-a164-4110-bcc3-1f6aa518f526\" (UID: \"3368d472-a164-4110-bcc3-1f6aa518f526\") "
Sep 18 18:58:20.473731 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.473692    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3368d472-a164-4110-bcc3-1f6aa518f526" (UID: "3368d472-a164-4110-bcc3-1f6aa518f526"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:58:20.473785 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.473703    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-home" (OuterVolumeSpecName: "home") pod "3368d472-a164-4110-bcc3-1f6aa518f526" (UID: "3368d472-a164-4110-bcc3-1f6aa518f526"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:58:20.473869 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.473799    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-model-cache" (OuterVolumeSpecName: "model-cache") pod "3368d472-a164-4110-bcc3-1f6aa518f526" (UID: "3368d472-a164-4110-bcc3-1f6aa518f526"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:58:20.475551 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.475528    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-dshm" (OuterVolumeSpecName: "dshm") pod "3368d472-a164-4110-bcc3-1f6aa518f526" (UID: "3368d472-a164-4110-bcc3-1f6aa518f526"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:58:20.475551 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.475545    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3368d472-a164-4110-bcc3-1f6aa518f526-kube-api-access-pqdgc" (OuterVolumeSpecName: "kube-api-access-pqdgc") pod "3368d472-a164-4110-bcc3-1f6aa518f526" (UID: "3368d472-a164-4110-bcc3-1f6aa518f526"). InnerVolumeSpecName "kube-api-access-pqdgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:58:20.475694 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.475589    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3368d472-a164-4110-bcc3-1f6aa518f526-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3368d472-a164-4110-bcc3-1f6aa518f526" (UID: "3368d472-a164-4110-bcc3-1f6aa518f526"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:58:20.511625 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.511600    2558 generic.go:356] "Generic (PLEG): container finished" podID="3368d472-a164-4110-bcc3-1f6aa518f526" containerID="11fdfedf447c76e2b5ae9812df051878c20583d7263e309f223592ecfc49cd0a" exitCode=0
Sep 18 18:58:20.511712 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.511638    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"
Sep 18 18:58:20.511712 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.511676    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l" event={"ID":"3368d472-a164-4110-bcc3-1f6aa518f526","Type":"ContainerDied","Data":"11fdfedf447c76e2b5ae9812df051878c20583d7263e309f223592ecfc49cd0a"}
Sep 18 18:58:20.511797 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.511717    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l" event={"ID":"3368d472-a164-4110-bcc3-1f6aa518f526","Type":"ContainerDied","Data":"aed256bc23088f13448699e3a82cabcb8c0d635ed0757b10f95903d1ba54880e"}
Sep 18 18:58:20.511797 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.511739    2558 scope.go:117] "RemoveContainer" containerID="11fdfedf447c76e2b5ae9812df051878c20583d7263e309f223592ecfc49cd0a"
Sep 18 18:58:20.520110 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.520094    2558 scope.go:117] "RemoveContainer" containerID="11fdfedf447c76e2b5ae9812df051878c20583d7263e309f223592ecfc49cd0a"
Sep 18 18:58:20.520353 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:58:20.520339    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11fdfedf447c76e2b5ae9812df051878c20583d7263e309f223592ecfc49cd0a\": container with ID starting with 11fdfedf447c76e2b5ae9812df051878c20583d7263e309f223592ecfc49cd0a not found: ID does not exist" containerID="11fdfedf447c76e2b5ae9812df051878c20583d7263e309f223592ecfc49cd0a"
Sep 18 18:58:20.520398 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.520360    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11fdfedf447c76e2b5ae9812df051878c20583d7263e309f223592ecfc49cd0a"} err="failed to get container status \"11fdfedf447c76e2b5ae9812df051878c20583d7263e309f223592ecfc49cd0a\": rpc error: code = NotFound desc = could not find container \"11fdfedf447c76e2b5ae9812df051878c20583d7263e309f223592ecfc49cd0a\": container with ID starting with 11fdfedf447c76e2b5ae9812df051878c20583d7263e309f223592ecfc49cd0a not found: ID does not exist"
Sep 18 18:58:20.533970 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.533949    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"]
Sep 18 18:58:20.538203 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.538185    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-897f61/test-model-897f61-kserve-78457fc9d7-5qn5l"]
Sep 18 18:58:20.574535 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.574514    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-tmp-dir\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:58:20.574535 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.574532    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-model-cache\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:58:20.574655 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.574542    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3368d472-a164-4110-bcc3-1f6aa518f526-tls-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:58:20.574655 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.574549    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-dshm\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:58:20.574655 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.574557    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqdgc\" (UniqueName: \"kubernetes.io/projected/3368d472-a164-4110-bcc3-1f6aa518f526-kube-api-access-pqdgc\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:58:20.574655 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.574566    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3368d472-a164-4110-bcc3-1f6aa518f526-home\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:58:20.653552 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.653520    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:20.724537 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:20.724462    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:21.320530 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:21.320490    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:21.845890 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:21.845858    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3368d472-a164-4110-bcc3-1f6aa518f526" path="/var/lib/kubelet/pods/3368d472-a164-4110-bcc3-1f6aa518f526/volumes"
Sep 18 18:58:22.009187 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:22.009141    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:22.628440 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:22.628403    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:22.770376 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:22.770346    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:22.868927 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:22.868887    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:23.019543 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:23.019514    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:23.703339 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:23.703306    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:24.446909 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:24.446873    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:25.174165 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:25.174127    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:25.828925 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:25.828896    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:26.410941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:26.410901    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:27.081495 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:27.081286    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:31.255896 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:31.255859    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:31.817392 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:31.817359    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"]
Sep 18 18:58:31.817640 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:31.817613    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8" podUID="74f726ce-8d89-4a20-865e-1c6e68fa9a7c" containerName="maas-api" containerID="cri-o://5f7b0418e6b618e1c129209bcd62f67d1645d9666076109b81f422c903cc86ab" gracePeriod=30
Sep 18 18:58:31.843431 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:31.843402    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:31.915036 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:31.914999    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:32.015063 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.015029    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:32.076903 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.076840    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:58:32.098561 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.098528    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:32.178934 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.178901    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-openshift-service-ca\") pod \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\" (UID: \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\") "
Sep 18 18:58:32.179109 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.178994    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-maas-api-tls\") pod \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\" (UID: \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\") "
Sep 18 18:58:32.179109 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.179051    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-metrics-certs\") pod \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\" (UID: \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\") "
Sep 18 18:58:32.179109 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.179092    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g75n5\" (UniqueName: \"kubernetes.io/projected/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-kube-api-access-g75n5\") pod \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\" (UID: \"74f726ce-8d89-4a20-865e-1c6e68fa9a7c\") "
Sep 18 18:58:32.179301 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.179274    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "74f726ce-8d89-4a20-865e-1c6e68fa9a7c" (UID: "74f726ce-8d89-4a20-865e-1c6e68fa9a7c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:58:32.179453 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.179430    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-openshift-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:58:32.181119 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.181089    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "74f726ce-8d89-4a20-865e-1c6e68fa9a7c" (UID: "74f726ce-8d89-4a20-865e-1c6e68fa9a7c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:58:32.181703 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.181679    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-kube-api-access-g75n5" (OuterVolumeSpecName: "kube-api-access-g75n5") pod "74f726ce-8d89-4a20-865e-1c6e68fa9a7c" (UID: "74f726ce-8d89-4a20-865e-1c6e68fa9a7c"). InnerVolumeSpecName "kube-api-access-g75n5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:58:32.181779 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.181753    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "74f726ce-8d89-4a20-865e-1c6e68fa9a7c" (UID: "74f726ce-8d89-4a20-865e-1c6e68fa9a7c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:58:32.280449 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.280421    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-maas-api-tls\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:58:32.280449 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.280447    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-metrics-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:58:32.280851 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.280457    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g75n5\" (UniqueName: \"kubernetes.io/projected/74f726ce-8d89-4a20-865e-1c6e68fa9a7c-kube-api-access-g75n5\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:58:32.441229 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.441149    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:32.542762 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.542724    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:32.580810 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.580778    2558 generic.go:356] "Generic (PLEG): container finished" podID="74f726ce-8d89-4a20-865e-1c6e68fa9a7c" containerID="5f7b0418e6b618e1c129209bcd62f67d1645d9666076109b81f422c903cc86ab" exitCode=0
Sep 18 18:58:32.580957 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.580841    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"
Sep 18 18:58:32.580957 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.580851    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8" event={"ID":"74f726ce-8d89-4a20-865e-1c6e68fa9a7c","Type":"ContainerDied","Data":"5f7b0418e6b618e1c129209bcd62f67d1645d9666076109b81f422c903cc86ab"}
Sep 18 18:58:32.580957 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.580915    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8" event={"ID":"74f726ce-8d89-4a20-865e-1c6e68fa9a7c","Type":"ContainerDied","Data":"2be3b7205be799e0b7b742ec215a99ac30bd7fba31718964c8842b0d612c8aab"}
Sep 18 18:58:32.580957 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.580945    2558 scope.go:117] "RemoveContainer" containerID="5f7b0418e6b618e1c129209bcd62f67d1645d9666076109b81f422c903cc86ab"
Sep 18 18:58:32.591171 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.591156    2558 scope.go:117] "RemoveContainer" containerID="5f7b0418e6b618e1c129209bcd62f67d1645d9666076109b81f422c903cc86ab"
Sep 18 18:58:32.591437 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:58:32.591419    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f7b0418e6b618e1c129209bcd62f67d1645d9666076109b81f422c903cc86ab\": container with ID starting with 5f7b0418e6b618e1c129209bcd62f67d1645d9666076109b81f422c903cc86ab not found: ID does not exist" containerID="5f7b0418e6b618e1c129209bcd62f67d1645d9666076109b81f422c903cc86ab"
Sep 18 18:58:32.591477 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.591447    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f7b0418e6b618e1c129209bcd62f67d1645d9666076109b81f422c903cc86ab"} err="failed to get container status \"5f7b0418e6b618e1c129209bcd62f67d1645d9666076109b81f422c903cc86ab\": rpc error: code = NotFound desc = could not find container \"5f7b0418e6b618e1c129209bcd62f67d1645d9666076109b81f422c903cc86ab\": container with ID starting with 5f7b0418e6b618e1c129209bcd62f67d1645d9666076109b81f422c903cc86ab not found: ID does not exist"
Sep 18 18:58:32.602908 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.602882    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"]
Sep 18 18:58:32.604841 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:32.604794    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-62de055a-7fd7c67969-dntw8"]
Sep 18 18:58:33.037310 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:33.037279    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:33.844908 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:33.844880    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74f726ce-8d89-4a20-865e-1c6e68fa9a7c" path="/var/lib/kubelet/pods/74f726ce-8d89-4a20-865e-1c6e68fa9a7c/volumes"
Sep 18 18:58:33.877069 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:33.877038    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:34.475209 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:34.475175    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:34.511785 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:34.511757    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn"]
Sep 18 18:58:34.511996 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:34.511965    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn" podUID="e54405a6-40d3-4ac3-a42e-3c33fc8ffffa" containerName="payload-pre-processing" containerID="cri-o://888da155b261bbbc492825b27b0a00e95db3863ae5f12b0147d215da3a96d42b" gracePeriod=30
Sep 18 18:58:34.574534 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:34.574488    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:34.664123 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:34.664083    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:34.719156 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:34.719112    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:34.773876 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:34.773851    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn"
Sep 18 18:58:34.806352 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:34.806327    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e54405a6-40d3-4ac3-a42e-3c33fc8ffffa-plugins-config-volume\") pod \"e54405a6-40d3-4ac3-a42e-3c33fc8ffffa\" (UID: \"e54405a6-40d3-4ac3-a42e-3c33fc8ffffa\") "
Sep 18 18:58:34.806480 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:34.806391    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wq8sn\" (UniqueName: \"kubernetes.io/projected/e54405a6-40d3-4ac3-a42e-3c33fc8ffffa-kube-api-access-wq8sn\") pod \"e54405a6-40d3-4ac3-a42e-3c33fc8ffffa\" (UID: \"e54405a6-40d3-4ac3-a42e-3c33fc8ffffa\") "
Sep 18 18:58:34.806641 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:34.806621    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e54405a6-40d3-4ac3-a42e-3c33fc8ffffa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e54405a6-40d3-4ac3-a42e-3c33fc8ffffa" (UID: "e54405a6-40d3-4ac3-a42e-3c33fc8ffffa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:58:34.808480 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:34.808448    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e54405a6-40d3-4ac3-a42e-3c33fc8ffffa-kube-api-access-wq8sn" (OuterVolumeSpecName: "kube-api-access-wq8sn") pod "e54405a6-40d3-4ac3-a42e-3c33fc8ffffa" (UID: "e54405a6-40d3-4ac3-a42e-3c33fc8ffffa"). InnerVolumeSpecName "kube-api-access-wq8sn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:58:34.907565 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:34.907537    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wq8sn\" (UniqueName: \"kubernetes.io/projected/e54405a6-40d3-4ac3-a42e-3c33fc8ffffa-kube-api-access-wq8sn\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:58:34.907565 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:34.907562    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e54405a6-40d3-4ac3-a42e-3c33fc8ffffa-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 18:58:35.036794 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:35.036710    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:35.137737 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:35.137704    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:35.601013 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:35.600980    2558 generic.go:356] "Generic (PLEG): container finished" podID="e54405a6-40d3-4ac3-a42e-3c33fc8ffffa" containerID="888da155b261bbbc492825b27b0a00e95db3863ae5f12b0147d215da3a96d42b" exitCode=0
Sep 18 18:58:35.601198 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:35.601030    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn"
Sep 18 18:58:35.601198 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:35.601063    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn" event={"ID":"e54405a6-40d3-4ac3-a42e-3c33fc8ffffa","Type":"ContainerDied","Data":"888da155b261bbbc492825b27b0a00e95db3863ae5f12b0147d215da3a96d42b"}
Sep 18 18:58:35.601198 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:35.601117    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn" event={"ID":"e54405a6-40d3-4ac3-a42e-3c33fc8ffffa","Type":"ContainerDied","Data":"ebf90464bbdcd96661ffeee281a1f20a369e88457102373eb78bf1e2f4316177"}
Sep 18 18:58:35.601198 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:35.601145    2558 scope.go:117] "RemoveContainer" containerID="888da155b261bbbc492825b27b0a00e95db3863ae5f12b0147d215da3a96d42b"
Sep 18 18:58:35.610492 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:35.610473    2558 scope.go:117] "RemoveContainer" containerID="888da155b261bbbc492825b27b0a00e95db3863ae5f12b0147d215da3a96d42b"
Sep 18 18:58:35.610752 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:58:35.610735    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"888da155b261bbbc492825b27b0a00e95db3863ae5f12b0147d215da3a96d42b\": container with ID starting with 888da155b261bbbc492825b27b0a00e95db3863ae5f12b0147d215da3a96d42b not found: ID does not exist" containerID="888da155b261bbbc492825b27b0a00e95db3863ae5f12b0147d215da3a96d42b"
Sep 18 18:58:35.610798 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:35.610762    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"888da155b261bbbc492825b27b0a00e95db3863ae5f12b0147d215da3a96d42b"} err="failed to get container status \"888da155b261bbbc492825b27b0a00e95db3863ae5f12b0147d215da3a96d42b\": rpc error: code = NotFound desc = could not find container \"888da155b261bbbc492825b27b0a00e95db3863ae5f12b0147d215da3a96d42b\": container with ID starting with 888da155b261bbbc492825b27b0a00e95db3863ae5f12b0147d215da3a96d42b not found: ID does not exist"
Sep 18 18:58:35.622591 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:35.622570    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn"]
Sep 18 18:58:35.626540 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:35.626517    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-897f61-55bfc459bb-jx7rn"]
Sep 18 18:58:35.846386 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:35.846354    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e54405a6-40d3-4ac3-a42e-3c33fc8ffffa" path="/var/lib/kubelet/pods/e54405a6-40d3-4ac3-a42e-3c33fc8ffffa/volumes"
Sep 18 18:58:35.923631 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:35.923554    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:36.829735 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:36.829694    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:38.509955 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:38.509916    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:38.609753 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:38.609716    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:38.784663 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:38.784578    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:39.113242 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:39.113162    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:39.213858 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:39.213805    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:39.357072 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:39.357033    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:40.872193 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:40.872151    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:40.972414 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:40.972385    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:41.151832 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:41.151738    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:41.710694 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:41.710661    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:42.006624 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:42.006587    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:42.105139 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:42.105111    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:45.332443 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:45.332404    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:45.545889 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:45.545856    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:45.674073 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:45.673996    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:45.842013 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:45.841982    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:45.945968 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:45.945875    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:46.045559 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:46.045529    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:46.464470 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:46.464440    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:47.363664 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:47.363632    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:47.466236 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:47.466176    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:47.753999 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:47.753957    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:47.894396 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:47.894346    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:47.990345 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:47.990312    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:48.012755 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.012683    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz"]
Sep 18 18:58:48.013151 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.013137    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74f726ce-8d89-4a20-865e-1c6e68fa9a7c" containerName="maas-api"
Sep 18 18:58:48.013196 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.013154    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="74f726ce-8d89-4a20-865e-1c6e68fa9a7c" containerName="maas-api"
Sep 18 18:58:48.013196 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.013174    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="850a7519-641a-4a10-9509-6bb7f462b0af" containerName="maas-api"
Sep 18 18:58:48.013196 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.013180    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="850a7519-641a-4a10-9509-6bb7f462b0af" containerName="maas-api"
Sep 18 18:58:48.013286 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.013199    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e54405a6-40d3-4ac3-a42e-3c33fc8ffffa" containerName="payload-pre-processing"
Sep 18 18:58:48.013286 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.013204    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e54405a6-40d3-4ac3-a42e-3c33fc8ffffa" containerName="payload-pre-processing"
Sep 18 18:58:48.013286 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.013213    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3368d472-a164-4110-bcc3-1f6aa518f526" containerName="main"
Sep 18 18:58:48.013286 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.013218    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3368d472-a164-4110-bcc3-1f6aa518f526" containerName="main"
Sep 18 18:58:48.013410 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.013306    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e54405a6-40d3-4ac3-a42e-3c33fc8ffffa" containerName="payload-pre-processing"
Sep 18 18:58:48.013410 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.013319    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3368d472-a164-4110-bcc3-1f6aa518f526" containerName="main"
Sep 18 18:58:48.013410 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.013328    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="74f726ce-8d89-4a20-865e-1c6e68fa9a7c" containerName="maas-api"
Sep 18 18:58:48.013410 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.013336    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="850a7519-641a-4a10-9509-6bb7f462b0af" containerName="maas-api"
Sep 18 18:58:48.016544 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.016529    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz"
Sep 18 18:58:48.020388 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.020366    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-298125-dockercfg-9xj86\""
Sep 18 18:58:48.020498 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.020385    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-298125\""
Sep 18 18:58:48.031588 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.031561    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz"]
Sep 18 18:58:48.132885 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.132846    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5hvf\" (UniqueName: \"kubernetes.io/projected/57e8a51c-44d7-43f8-ae87-2e61e74ef886-kube-api-access-m5hvf\") pod \"payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz\" (UID: \"57e8a51c-44d7-43f8-ae87-2e61e74ef886\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz"
Sep 18 18:58:48.133075 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.132960    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57e8a51c-44d7-43f8-ae87-2e61e74ef886-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz\" (UID: \"57e8a51c-44d7-43f8-ae87-2e61e74ef886\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz"
Sep 18 18:58:48.234398 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.234361    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57e8a51c-44d7-43f8-ae87-2e61e74ef886-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz\" (UID: \"57e8a51c-44d7-43f8-ae87-2e61e74ef886\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz"
Sep 18 18:58:48.234545 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.234471    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5hvf\" (UniqueName: \"kubernetes.io/projected/57e8a51c-44d7-43f8-ae87-2e61e74ef886-kube-api-access-m5hvf\") pod \"payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz\" (UID: \"57e8a51c-44d7-43f8-ae87-2e61e74ef886\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz"
Sep 18 18:58:48.235152 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.235132    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57e8a51c-44d7-43f8-ae87-2e61e74ef886-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz\" (UID: \"57e8a51c-44d7-43f8-ae87-2e61e74ef886\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz"
Sep 18 18:58:48.243788 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.243765    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5hvf\" (UniqueName: \"kubernetes.io/projected/57e8a51c-44d7-43f8-ae87-2e61e74ef886-kube-api-access-m5hvf\") pod \"payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz\" (UID: \"57e8a51c-44d7-43f8-ae87-2e61e74ef886\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz"
Sep 18 18:58:48.327053 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.326989    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz"
Sep 18 18:58:48.601622 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.601597    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz"]
Sep 18 18:58:48.603282 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:58:48.603249    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57e8a51c_44d7_43f8_ae87_2e61e74ef886.slice/crio-8566727bdfd1c98161bcd3bf2486dddfee9980f9fc07f1fcde7ceac4870b67a3 WatchSource:0}: Error finding container 8566727bdfd1c98161bcd3bf2486dddfee9980f9fc07f1fcde7ceac4870b67a3: Status 404 returned error can't find the container with id 8566727bdfd1c98161bcd3bf2486dddfee9980f9fc07f1fcde7ceac4870b67a3
Sep 18 18:58:48.668942 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:48.668919    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz" event={"ID":"57e8a51c-44d7-43f8-ae87-2e61e74ef886","Type":"ContainerStarted","Data":"8566727bdfd1c98161bcd3bf2486dddfee9980f9fc07f1fcde7ceac4870b67a3"}
Sep 18 18:58:49.676430 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:49.676387    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz" event={"ID":"57e8a51c-44d7-43f8-ae87-2e61e74ef886","Type":"ContainerStarted","Data":"098c80549be205fc4bdf40964a5ab775eb2bf57d246e8c67a0e006fd608fb4a8"}
Sep 18 18:58:49.676870 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:49.676520    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz"
Sep 18 18:58:49.703709 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:49.703665    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz" podStartSLOduration=2.703651964 podStartE2EDuration="2.703651964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:58:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:58:49.702457728 +0000 UTC m=+1165.900941943" watchObservedRunningTime="2026-09-18 18:58:49.703651964 +0000 UTC m=+1165.902136168"
Sep 18 18:58:51.160214 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:51.160182    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:52.643300 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:52.643269    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:52.896463 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:52.896373    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:53.179941 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:53.179859    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:53.869014 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:53.868980    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:54.245071 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:54.245039    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:54.333163 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:54.333129    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:56.834443 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:56.834388    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:57.421924 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:57.421884    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:57.585505 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:57.585460    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:57.677664 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:57.677578    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:58.437023 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:58.436983    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:59.139708 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:59.139677    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:58:59.927353 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:59.927315    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"]
Sep 18 18:58:59.930737 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:59.930719    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 18:58:59.935204 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:59.935183    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-880bee-serving-cert\""
Sep 18 18:58:59.943894 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:59.943870    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"]
Sep 18 18:58:59.946770 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:59.946742    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3d18575-7648-42f0-a048-ce5fe38237e9-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s\" (UID: \"d3d18575-7648-42f0-a048-ce5fe38237e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 18:58:59.946928 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:59.946795    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgh5t\" (UniqueName: \"kubernetes.io/projected/d3d18575-7648-42f0-a048-ce5fe38237e9-kube-api-access-wgh5t\") pod \"maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s\" (UID: \"d3d18575-7648-42f0-a048-ce5fe38237e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 18:58:59.947003 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:59.946924    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3d18575-7648-42f0-a048-ce5fe38237e9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s\" (UID: \"d3d18575-7648-42f0-a048-ce5fe38237e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 18:58:59.947064 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:59.947002    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3d18575-7648-42f0-a048-ce5fe38237e9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s\" (UID: \"d3d18575-7648-42f0-a048-ce5fe38237e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 18:58:59.965607 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:58:59.965582    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:00.047592 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.047561    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3d18575-7648-42f0-a048-ce5fe38237e9-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s\" (UID: \"d3d18575-7648-42f0-a048-ce5fe38237e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 18:59:00.047778 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.047605    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgh5t\" (UniqueName: \"kubernetes.io/projected/d3d18575-7648-42f0-a048-ce5fe38237e9-kube-api-access-wgh5t\") pod \"maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s\" (UID: \"d3d18575-7648-42f0-a048-ce5fe38237e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 18:59:00.047880 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.047810    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3d18575-7648-42f0-a048-ce5fe38237e9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s\" (UID: \"d3d18575-7648-42f0-a048-ce5fe38237e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 18:59:00.047957 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.047936    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3d18575-7648-42f0-a048-ce5fe38237e9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s\" (UID: \"d3d18575-7648-42f0-a048-ce5fe38237e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 18:59:00.048022 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:59:00.047942    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-serving-cert: secret "maas-api-e2e-shared-a-w1-880bee-serving-cert" not found
Sep 18 18:59:00.048104 ip-10-0-130-118 kubenswrapper[2558]: E0918 18:59:00.048092    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3d18575-7648-42f0-a048-ce5fe38237e9-maas-api-tls podName:d3d18575-7648-42f0-a048-ce5fe38237e9 nodeName:}" failed. No retries permitted until 2026-09-18 18:59:00.548074158 +0000 UTC m=+1176.746558358 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d3d18575-7648-42f0-a048-ce5fe38237e9-maas-api-tls") pod "maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s" (UID: "d3d18575-7648-42f0-a048-ce5fe38237e9") : secret "maas-api-e2e-shared-a-w1-880bee-serving-cert" not found
Sep 18 18:59:00.048554 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.048533    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3d18575-7648-42f0-a048-ce5fe38237e9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s\" (UID: \"d3d18575-7648-42f0-a048-ce5fe38237e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 18:59:00.050093 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.050066    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3d18575-7648-42f0-a048-ce5fe38237e9-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s\" (UID: \"d3d18575-7648-42f0-a048-ce5fe38237e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 18:59:00.057338 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.057316    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgh5t\" (UniqueName: \"kubernetes.io/projected/d3d18575-7648-42f0-a048-ce5fe38237e9-kube-api-access-wgh5t\") pod \"maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s\" (UID: \"d3d18575-7648-42f0-a048-ce5fe38237e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 18:59:00.061258 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.061234    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:00.236137 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.236053    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:00.341029 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.340995    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:00.554123 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.554086    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3d18575-7648-42f0-a048-ce5fe38237e9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s\" (UID: \"d3d18575-7648-42f0-a048-ce5fe38237e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 18:59:00.556559 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.556530    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3d18575-7648-42f0-a048-ce5fe38237e9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s\" (UID: \"d3d18575-7648-42f0-a048-ce5fe38237e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 18:59:00.568213 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.568176    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:00.658535 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.658498    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:00.683674 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.683643    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz"
Sep 18 18:59:00.699604 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.699577    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg"]
Sep 18 18:59:00.703503 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.703486    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg"
Sep 18 18:59:00.706793 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.706774    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-880bee\""
Sep 18 18:59:00.722106 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.722079    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg"]
Sep 18 18:59:00.755267 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.755232    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df43d446-cb9a-4201-96e9-f40f93b98955-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg\" (UID: \"df43d446-cb9a-4201-96e9-f40f93b98955\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg"
Sep 18 18:59:00.755434 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.755322    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-669fc\" (UniqueName: \"kubernetes.io/projected/df43d446-cb9a-4201-96e9-f40f93b98955-kube-api-access-669fc\") pod \"payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg\" (UID: \"df43d446-cb9a-4201-96e9-f40f93b98955\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg"
Sep 18 18:59:00.841030 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.840952    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 18:59:00.856979 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.856949    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df43d446-cb9a-4201-96e9-f40f93b98955-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg\" (UID: \"df43d446-cb9a-4201-96e9-f40f93b98955\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg"
Sep 18 18:59:00.857130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.857108    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-669fc\" (UniqueName: \"kubernetes.io/projected/df43d446-cb9a-4201-96e9-f40f93b98955-kube-api-access-669fc\") pod \"payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg\" (UID: \"df43d446-cb9a-4201-96e9-f40f93b98955\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg"
Sep 18 18:59:00.857666 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.857645    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df43d446-cb9a-4201-96e9-f40f93b98955-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg\" (UID: \"df43d446-cb9a-4201-96e9-f40f93b98955\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg"
Sep 18 18:59:00.876516 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:00.876479    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-669fc\" (UniqueName: \"kubernetes.io/projected/df43d446-cb9a-4201-96e9-f40f93b98955-kube-api-access-669fc\") pod \"payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg\" (UID: \"df43d446-cb9a-4201-96e9-f40f93b98955\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg"
Sep 18 18:59:01.014720 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:01.014684    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg"
Sep 18 18:59:01.055465 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:01.055429    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:01.150528 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:01.150505    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"]
Sep 18 18:59:01.152612 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:59:01.152586    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3d18575_7648_42f0_a048_ce5fe38237e9.slice/crio-9b626a269bef65560266e8454b90811d9423ed23ddfa6cff1788e4c1bba82a0c WatchSource:0}: Error finding container 9b626a269bef65560266e8454b90811d9423ed23ddfa6cff1788e4c1bba82a0c: Status 404 returned error can't find the container with id 9b626a269bef65560266e8454b90811d9423ed23ddfa6cff1788e4c1bba82a0c
Sep 18 18:59:01.172812 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:01.172780    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:01.298273 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:01.298251    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg"]
Sep 18 18:59:01.300328 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:59:01.300298    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf43d446_cb9a_4201_96e9_f40f93b98955.slice/crio-fb05d7f51b26fad05835cf5a73399fa2c89680bba92ab81bfd721b6c0373f55f WatchSource:0}: Error finding container fb05d7f51b26fad05835cf5a73399fa2c89680bba92ab81bfd721b6c0373f55f: Status 404 returned error can't find the container with id fb05d7f51b26fad05835cf5a73399fa2c89680bba92ab81bfd721b6c0373f55f
Sep 18 18:59:01.557784 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:01.557748    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:01.742345 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:01.742310    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg" event={"ID":"df43d446-cb9a-4201-96e9-f40f93b98955","Type":"ContainerStarted","Data":"b940fedc8adefae222cc9dced48a015b0557e44b7b5d9f49f6e491c9eeab93c9"}
Sep 18 18:59:01.742488 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:01.742398    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg" event={"ID":"df43d446-cb9a-4201-96e9-f40f93b98955","Type":"ContainerStarted","Data":"fb05d7f51b26fad05835cf5a73399fa2c89680bba92ab81bfd721b6c0373f55f"}
Sep 18 18:59:01.742488 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:01.742424    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg"
Sep 18 18:59:01.743880 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:01.743856    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s" event={"ID":"d3d18575-7648-42f0-a048-ce5fe38237e9","Type":"ContainerStarted","Data":"9b626a269bef65560266e8454b90811d9423ed23ddfa6cff1788e4c1bba82a0c"}
Sep 18 18:59:02.251262 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:02.251226    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:02.752537 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:02.752506    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s" event={"ID":"d3d18575-7648-42f0-a048-ce5fe38237e9","Type":"ContainerStarted","Data":"c2c6655a0070039435fb6d89c7f41b732f58ccf945b0780c7200613a91e93b50"}
Sep 18 18:59:02.775481 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:02.775433    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg" podStartSLOduration=2.775417727 podStartE2EDuration="2.775417727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:59:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:01.78344877 +0000 UTC m=+1177.981932986" watchObservedRunningTime="2026-09-18 18:59:02.775417727 +0000 UTC m=+1178.973901931"
Sep 18 18:59:02.776778 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:02.776748    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s" podStartSLOduration=3.2653683239999998 podStartE2EDuration="3.776735383s" totalImagesPullingTime="511.367059ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:02.775482123 +0000 UTC m=+1178.973966347" watchObservedRunningTime="2026-09-18 18:59:02.776735383 +0000 UTC m=+1178.975219587"
Sep 18 18:59:03.194380 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:03.194337    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:03.759459 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:03.759431    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 18:59:04.095541 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:04.095450    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:04.637290 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:04.637253    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:04.765259 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:04.765221    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:05.364137 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:05.364106    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:06.608458 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:06.608415    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:06.739353 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:06.739309    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:07.234373 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:07.234337    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:07.345388 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:07.345356    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:07.444970 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:07.444940    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld"]
Sep 18 18:59:07.448711 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:07.448693    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld"
Sep 18 18:59:07.455167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:07.454943    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-a6d509-dockercfg-k8ljh\""
Sep 18 18:59:07.455167 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:07.455107    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-a6d509\""
Sep 18 18:59:07.466993 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:07.466970    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld"]
Sep 18 18:59:07.528634 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:07.528605    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/358e8e15-d5aa-45e0-929c-05f7f90c2b0a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld\" (UID: \"358e8e15-d5aa-45e0-929c-05f7f90c2b0a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld"
Sep 18 18:59:07.528769 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:07.528642    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgnbd\" (UniqueName: \"kubernetes.io/projected/358e8e15-d5aa-45e0-929c-05f7f90c2b0a-kube-api-access-wgnbd\") pod \"payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld\" (UID: \"358e8e15-d5aa-45e0-929c-05f7f90c2b0a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld"
Sep 18 18:59:07.629367 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:07.629338    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgnbd\" (UniqueName: \"kubernetes.io/projected/358e8e15-d5aa-45e0-929c-05f7f90c2b0a-kube-api-access-wgnbd\") pod \"payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld\" (UID: \"358e8e15-d5aa-45e0-929c-05f7f90c2b0a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld"
Sep 18 18:59:07.629727 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:07.629487    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/358e8e15-d5aa-45e0-929c-05f7f90c2b0a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld\" (UID: \"358e8e15-d5aa-45e0-929c-05f7f90c2b0a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld"
Sep 18 18:59:07.630147 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:07.630128    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/358e8e15-d5aa-45e0-929c-05f7f90c2b0a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld\" (UID: \"358e8e15-d5aa-45e0-929c-05f7f90c2b0a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld"
Sep 18 18:59:07.640461 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:07.640436    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgnbd\" (UniqueName: \"kubernetes.io/projected/358e8e15-d5aa-45e0-929c-05f7f90c2b0a-kube-api-access-wgnbd\") pod \"payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld\" (UID: \"358e8e15-d5aa-45e0-929c-05f7f90c2b0a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld"
Sep 18 18:59:07.758944 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:07.758912    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld"
Sep 18 18:59:07.877881 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:07.877847    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:08.044555 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:08.044531    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld"]
Sep 18 18:59:08.046396 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:59:08.046361    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod358e8e15_d5aa_45e0_929c_05f7f90c2b0a.slice/crio-b7bb445763f2648b396161878135c5dcba9593b5a6d8b5df0ed3428224ca6fbb WatchSource:0}: Error finding container b7bb445763f2648b396161878135c5dcba9593b5a6d8b5df0ed3428224ca6fbb: Status 404 returned error can't find the container with id b7bb445763f2648b396161878135c5dcba9593b5a6d8b5df0ed3428224ca6fbb
Sep 18 18:59:08.791314 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:08.791274    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:08.794592 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:08.794553    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld" event={"ID":"358e8e15-d5aa-45e0-929c-05f7f90c2b0a","Type":"ContainerStarted","Data":"6864a5b0fe8e3a5261ff59032f19cd6d1abe752bd732c022a824c29123e665bc"}
Sep 18 18:59:08.794714 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:08.794616    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld" event={"ID":"358e8e15-d5aa-45e0-929c-05f7f90c2b0a","Type":"ContainerStarted","Data":"b7bb445763f2648b396161878135c5dcba9593b5a6d8b5df0ed3428224ca6fbb"}
Sep 18 18:59:08.794714 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:08.794669    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld"
Sep 18 18:59:08.819707 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:08.819656    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld" podStartSLOduration=1.819639246 podStartE2EDuration="1.819639246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:59:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:08.817549268 +0000 UTC m=+1185.016033490" watchObservedRunningTime="2026-09-18 18:59:08.819639246 +0000 UTC m=+1185.018123450"
Sep 18 18:59:09.756733 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:09.756701    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:09.771591 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:09.771569    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 18:59:10.440021 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:10.439983    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:11.515670 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:11.515637    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:12.754706 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:12.754678    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg"
Sep 18 18:59:19.803235 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:19.803200    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld"
Sep 18 18:59:21.150444 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:21.150404    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:22.451899 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:22.451862    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:22.618505 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:22.618441    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:22.950464 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:22.950432    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:23.512728 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:23.512692    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:23.679368 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:23.679336    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:23.802876 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:23.802771    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:24.413566 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:24.413534    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:24.513707 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:24.513667    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:25.616059 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:25.616027    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:25.834132 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:25.834101    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bwsg_636f0979-fe2e-49fd-86d6-31bd7961e736/ovn-acl-logging/0.log"
Sep 18 18:59:25.846896 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:25.846874    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bwsg_636f0979-fe2e-49fd-86d6-31bd7961e736/ovn-acl-logging/0.log"
Sep 18 18:59:26.436532 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:26.436501    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:26.738371 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:26.738279    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:26.836670 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:26.836630    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:27.289735 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:27.289697    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:28.145263 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:28.145229    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:28.891721 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:28.891684    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:29.568085 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.568049    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"]
Sep 18 18:59:29.573103 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.573082    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"
Sep 18 18:59:29.576193 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.576170    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-351045-serving-cert\""
Sep 18 18:59:29.584147 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.584119    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"]
Sep 18 18:59:29.637165 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.637129    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:29.646642 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.646616    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e1d1183-db92-40da-89f1-6b1d8302411f-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh\" (UID: \"0e1d1183-db92-40da-89f1-6b1d8302411f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"
Sep 18 18:59:29.646787 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.646660    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e1d1183-db92-40da-89f1-6b1d8302411f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh\" (UID: \"0e1d1183-db92-40da-89f1-6b1d8302411f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"
Sep 18 18:59:29.646787 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.646681    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1d1183-db92-40da-89f1-6b1d8302411f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh\" (UID: \"0e1d1183-db92-40da-89f1-6b1d8302411f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"
Sep 18 18:59:29.646787 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.646736    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rwrg\" (UniqueName: \"kubernetes.io/projected/0e1d1183-db92-40da-89f1-6b1d8302411f-kube-api-access-7rwrg\") pod \"maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh\" (UID: \"0e1d1183-db92-40da-89f1-6b1d8302411f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"
Sep 18 18:59:29.747637 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.747597    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rwrg\" (UniqueName: \"kubernetes.io/projected/0e1d1183-db92-40da-89f1-6b1d8302411f-kube-api-access-7rwrg\") pod \"maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh\" (UID: \"0e1d1183-db92-40da-89f1-6b1d8302411f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"
Sep 18 18:59:29.747857 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.747765    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e1d1183-db92-40da-89f1-6b1d8302411f-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh\" (UID: \"0e1d1183-db92-40da-89f1-6b1d8302411f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"
Sep 18 18:59:29.747857 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.747809    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e1d1183-db92-40da-89f1-6b1d8302411f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh\" (UID: \"0e1d1183-db92-40da-89f1-6b1d8302411f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"
Sep 18 18:59:29.747999 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.747880    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1d1183-db92-40da-89f1-6b1d8302411f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh\" (UID: \"0e1d1183-db92-40da-89f1-6b1d8302411f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"
Sep 18 18:59:29.748746 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.748724    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1d1183-db92-40da-89f1-6b1d8302411f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh\" (UID: \"0e1d1183-db92-40da-89f1-6b1d8302411f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"
Sep 18 18:59:29.748871 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.748741    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:29.750322 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.750304    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e1d1183-db92-40da-89f1-6b1d8302411f-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh\" (UID: \"0e1d1183-db92-40da-89f1-6b1d8302411f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"
Sep 18 18:59:29.750406 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.750354    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e1d1183-db92-40da-89f1-6b1d8302411f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh\" (UID: \"0e1d1183-db92-40da-89f1-6b1d8302411f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"
Sep 18 18:59:29.757282 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.757260    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rwrg\" (UniqueName: \"kubernetes.io/projected/0e1d1183-db92-40da-89f1-6b1d8302411f-kube-api-access-7rwrg\") pod \"maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh\" (UID: \"0e1d1183-db92-40da-89f1-6b1d8302411f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"
Sep 18 18:59:29.837704 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.837628    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:29.884995 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:29.884956    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"
Sep 18 18:59:30.176596 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.176569    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"]
Sep 18 18:59:30.178143 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:59:30.178118    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e1d1183_db92_40da_89f1_6b1d8302411f.slice/crio-0f33bd021eecf38f31971d637cb1dff7767835163a4c08b3061cc9ccd631c874 WatchSource:0}: Error finding container 0f33bd021eecf38f31971d637cb1dff7767835163a4c08b3061cc9ccd631c874: Status 404 returned error can't find the container with id 0f33bd021eecf38f31971d637cb1dff7767835163a4c08b3061cc9ccd631c874
Sep 18 18:59:30.317197 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.317163    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:30.433856 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.433761    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:30.463285 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.463251    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj"]
Sep 18 18:59:30.468370 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.468347    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj"
Sep 18 18:59:30.471899 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.471883    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-351045\""
Sep 18 18:59:30.479316 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.479295    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj"]
Sep 18 18:59:30.554228 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.554199    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac14ae1c-9d31-465c-8de0-a6ab51b5e00e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj\" (UID: \"ac14ae1c-9d31-465c-8de0-a6ab51b5e00e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj"
Sep 18 18:59:30.554404 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.554238    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj2j5\" (UniqueName: \"kubernetes.io/projected/ac14ae1c-9d31-465c-8de0-a6ab51b5e00e-kube-api-access-jj2j5\") pod \"payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj\" (UID: \"ac14ae1c-9d31-465c-8de0-a6ab51b5e00e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj"
Sep 18 18:59:30.655050 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.655014    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac14ae1c-9d31-465c-8de0-a6ab51b5e00e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj\" (UID: \"ac14ae1c-9d31-465c-8de0-a6ab51b5e00e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj"
Sep 18 18:59:30.655474 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.655070    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj2j5\" (UniqueName: \"kubernetes.io/projected/ac14ae1c-9d31-465c-8de0-a6ab51b5e00e-kube-api-access-jj2j5\") pod \"payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj\" (UID: \"ac14ae1c-9d31-465c-8de0-a6ab51b5e00e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj"
Sep 18 18:59:30.655687 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.655667    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac14ae1c-9d31-465c-8de0-a6ab51b5e00e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj\" (UID: \"ac14ae1c-9d31-465c-8de0-a6ab51b5e00e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj"
Sep 18 18:59:30.663393 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.663369    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj2j5\" (UniqueName: \"kubernetes.io/projected/ac14ae1c-9d31-465c-8de0-a6ab51b5e00e-kube-api-access-jj2j5\") pod \"payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj\" (UID: \"ac14ae1c-9d31-465c-8de0-a6ab51b5e00e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj"
Sep 18 18:59:30.780301 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.780277    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj"
Sep 18 18:59:30.785049 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.785007    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:30.919354 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.919323    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh" event={"ID":"0e1d1183-db92-40da-89f1-6b1d8302411f","Type":"ContainerStarted","Data":"ddaf149c05ededed0b825d5785f5eff59189371199e51b69dd4485870fcac19e"}
Sep 18 18:59:30.919492 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.919371    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh" event={"ID":"0e1d1183-db92-40da-89f1-6b1d8302411f","Type":"ContainerStarted","Data":"0f33bd021eecf38f31971d637cb1dff7767835163a4c08b3061cc9ccd631c874"}
Sep 18 18:59:30.919492 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.919468    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"
Sep 18 18:59:30.946687 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:30.946642    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh" podStartSLOduration=1.3519942870000001 podStartE2EDuration="1.94662664s" totalImagesPullingTime="594.632353ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:59:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:30.944653524 +0000 UTC m=+1207.143137751" watchObservedRunningTime="2026-09-18 18:59:30.94662664 +0000 UTC m=+1207.145110871"
Sep 18 18:59:31.067455 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:31.067424    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj"]
Sep 18 18:59:31.067864 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:59:31.067834    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac14ae1c_9d31_465c_8de0_a6ab51b5e00e.slice/crio-c13f2d38a1499bb837d86494c3c7fa606aad6d607c5d996ed21d1a3c85eb075b WatchSource:0}: Error finding container c13f2d38a1499bb837d86494c3c7fa606aad6d607c5d996ed21d1a3c85eb075b: Status 404 returned error can't find the container with id c13f2d38a1499bb837d86494c3c7fa606aad6d607c5d996ed21d1a3c85eb075b
Sep 18 18:59:31.672439 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:31.672395    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:31.927066 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:31.926968    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj" event={"ID":"ac14ae1c-9d31-465c-8de0-a6ab51b5e00e","Type":"ContainerStarted","Data":"be2ae9be566f9957c410ae46e791685a2e67cd6f8586c893e6037b8255b14f79"}
Sep 18 18:59:31.927066 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:31.927031    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj" event={"ID":"ac14ae1c-9d31-465c-8de0-a6ab51b5e00e","Type":"ContainerStarted","Data":"c13f2d38a1499bb837d86494c3c7fa606aad6d607c5d996ed21d1a3c85eb075b"}
Sep 18 18:59:31.927295 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:31.927207    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj"
Sep 18 18:59:31.955424 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:31.955380    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj" podStartSLOduration=1.955364611 podStartE2EDuration="1.955364611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:59:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:31.954439401 +0000 UTC m=+1208.152923629" watchObservedRunningTime="2026-09-18 18:59:31.955364611 +0000 UTC m=+1208.153848867"
Sep 18 18:59:32.649059 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:32.649020    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:36.932720 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:36.932688    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"
Sep 18 18:59:42.935646 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:42.935609    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj"
Sep 18 18:59:51.673022 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:51.672981    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:53.067731 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.067693    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:53.542398 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.542363    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:53.757533 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.757497    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"]
Sep 18 18:59:53.760742 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.760721    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:53.764420 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.764396    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-298125\"/\"auth-test-model-298125-kserve-self-signed-certs\""
Sep 18 18:59:53.764548 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.764432    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-298125\"/\"openshift-service-ca.crt\""
Sep 18 18:59:53.764548 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.764432    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-298125\"/\"default-dockercfg-tg8lq\""
Sep 18 18:59:53.764548 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.764432    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-298125\"/\"kube-root-ca.crt\""
Sep 18 18:59:53.771754 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.771735    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"]
Sep 18 18:59:53.873880 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.873778    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:53.883969 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.883948    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-tmp-dir\") pod \"auth-test-model-298125-kserve-7b5998f9c9-m4ch2\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") " pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:53.884053 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.883978    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-dshm\") pod \"auth-test-model-298125-kserve-7b5998f9c9-m4ch2\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") " pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:53.884053 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.884012    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5de8fae-16b2-4b47-9941-9d436dcc2efd-tls-certs\") pod \"auth-test-model-298125-kserve-7b5998f9c9-m4ch2\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") " pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:53.884129 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.884088    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-model-cache\") pod \"auth-test-model-298125-kserve-7b5998f9c9-m4ch2\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") " pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:53.884178 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.884126    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-home\") pod \"auth-test-model-298125-kserve-7b5998f9c9-m4ch2\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") " pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:53.884311 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.884266    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvbxm\" (UniqueName: \"kubernetes.io/projected/b5de8fae-16b2-4b47-9941-9d436dcc2efd-kube-api-access-jvbxm\") pod \"auth-test-model-298125-kserve-7b5998f9c9-m4ch2\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") " pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:53.984979 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.984948    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-tmp-dir\") pod \"auth-test-model-298125-kserve-7b5998f9c9-m4ch2\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") " pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:53.985162 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.984988    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-dshm\") pod \"auth-test-model-298125-kserve-7b5998f9c9-m4ch2\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") " pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:53.985162 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.985033    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5de8fae-16b2-4b47-9941-9d436dcc2efd-tls-certs\") pod \"auth-test-model-298125-kserve-7b5998f9c9-m4ch2\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") " pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:53.985162 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.985081    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-model-cache\") pod \"auth-test-model-298125-kserve-7b5998f9c9-m4ch2\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") " pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:53.985162 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.985118    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-home\") pod \"auth-test-model-298125-kserve-7b5998f9c9-m4ch2\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") " pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:53.985395 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.985207    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvbxm\" (UniqueName: \"kubernetes.io/projected/b5de8fae-16b2-4b47-9941-9d436dcc2efd-kube-api-access-jvbxm\") pod \"auth-test-model-298125-kserve-7b5998f9c9-m4ch2\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") " pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:53.985487 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.985456    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-tmp-dir\") pod \"auth-test-model-298125-kserve-7b5998f9c9-m4ch2\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") " pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:53.985611 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.985559    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-home\") pod \"auth-test-model-298125-kserve-7b5998f9c9-m4ch2\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") " pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:53.985705 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.985687    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-model-cache\") pod \"auth-test-model-298125-kserve-7b5998f9c9-m4ch2\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") " pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:53.987309 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.987282    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-dshm\") pod \"auth-test-model-298125-kserve-7b5998f9c9-m4ch2\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") " pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:53.987587 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.987570    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5de8fae-16b2-4b47-9941-9d436dcc2efd-tls-certs\") pod \"auth-test-model-298125-kserve-7b5998f9c9-m4ch2\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") " pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:53.993219 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:53.993197    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvbxm\" (UniqueName: \"kubernetes.io/projected/b5de8fae-16b2-4b47-9941-9d436dcc2efd-kube-api-access-jvbxm\") pod \"auth-test-model-298125-kserve-7b5998f9c9-m4ch2\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") " pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:54.071255 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:54.071225    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 18:59:54.272511 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:54.272467    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:54.354608 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:54.354568    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"]
Sep 18 18:59:54.360981 ip-10-0-130-118 kubenswrapper[2558]: W0918 18:59:54.360958    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5de8fae_16b2_4b47_9941_9d436dcc2efd.slice/crio-9dab51324a0e6892aa2bb64e0384cb654c748718a11d111afbc5d6631bcbd87e WatchSource:0}: Error finding container 9dab51324a0e6892aa2bb64e0384cb654c748718a11d111afbc5d6631bcbd87e: Status 404 returned error can't find the container with id 9dab51324a0e6892aa2bb64e0384cb654c748718a11d111afbc5d6631bcbd87e
Sep 18 18:59:55.052953 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:55.052913    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2" event={"ID":"b5de8fae-16b2-4b47-9941-9d436dcc2efd","Type":"ContainerStarted","Data":"0e7fdf0307c04f5c1e61b29f51ba1d9e02bb4161bb50e6b7763754cfa6f1e3cf"}
Sep 18 18:59:55.053130 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:55.052962    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2" event={"ID":"b5de8fae-16b2-4b47-9941-9d436dcc2efd","Type":"ContainerStarted","Data":"9dab51324a0e6892aa2bb64e0384cb654c748718a11d111afbc5d6631bcbd87e"}
Sep 18 18:59:55.074433 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:55.074389    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2" podStartSLOduration=2.074374411 podStartE2EDuration="2.074374411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:59:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:55.073190662 +0000 UTC m=+1231.271674895" watchObservedRunningTime="2026-09-18 18:59:55.074374411 +0000 UTC m=+1231.272858614"
Sep 18 18:59:55.080986 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:55.080949    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:55.918759 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:55.918723    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 18:59:56.834763 ip-10-0-130-118 kubenswrapper[2558]: I0918 18:59:56.834722    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:00.203363 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:00.203327    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-351045-29829300-x9gqz"]
Sep 18 19:00:00.207121 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:00.207104    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-351045-29829300-x9gqz"
Sep 18 19:00:00.209644 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:00.209618    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xvbnc\""
Sep 18 19:00:00.214681 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:00.214659    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-351045-29829300-x9gqz"]
Sep 18 19:00:00.220204 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:00.220185    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-351045-29829300-x9gqz"
Sep 18 19:00:00.497874 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:00.497849    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-351045-29829300-x9gqz"]
Sep 18 19:00:00.499409 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:00:00.499384    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ab9b3e9_64b0_4955_a7a4_d84cf9ca6518.slice/crio-56b1d72b7ca5aec67e9a33808b0f9b06dc4595a108b8d4af97806eea18d4b7a0 WatchSource:0}: Error finding container 56b1d72b7ca5aec67e9a33808b0f9b06dc4595a108b8d4af97806eea18d4b7a0: Status 404 returned error can't find the container with id 56b1d72b7ca5aec67e9a33808b0f9b06dc4595a108b8d4af97806eea18d4b7a0
Sep 18 19:00:01.085114 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:01.085084    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-351045-29829300-x9gqz" event={"ID":"0ab9b3e9-64b0-4955-a7a4-d84cf9ca6518","Type":"ContainerStarted","Data":"56b1d72b7ca5aec67e9a33808b0f9b06dc4595a108b8d4af97806eea18d4b7a0"}
Sep 18 19:00:01.375565 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:01.375456    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:01.563665 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:01.563631    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:02.723678 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:02.723652    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:04.072215 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:04.072174    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 19:00:04.072612 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:04.072233    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 19:00:04.086406 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:04.086378    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 19:00:04.114431 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:04.114407    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 19:00:04.258467 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:04.258431    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:05.049072 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:05.048996    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:05.154082 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:05.154049    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:06.690210 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:06.690168    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:06.791491 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:06.791456    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:07.656325 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:07.656284    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:09.058067 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:09.058029    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:09.346763 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:09.346684    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:10.097001 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:10.096968    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:10.197320 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:10.197286    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:10.332432 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:10.332392    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:10.712024 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:10.711993    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:10.808429 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:10.808397    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:10.865504 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:10.865475    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss"]
Sep 18 19:00:10.870365 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:10.870345    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss"
Sep 18 19:00:10.878704 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:10.878687    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-25dc6358-dockercfg-9cst8\""
Sep 18 19:00:10.879124 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:10.879105    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-25dc6358\""
Sep 18 19:00:10.886202 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:10.886180    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss"]
Sep 18 19:00:10.910422 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:10.910390    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t"]
Sep 18 19:00:10.914051 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:10.914033    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t"
Sep 18 19:00:10.939540 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:10.939512    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t"]
Sep 18 19:00:10.952283 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:10.952258    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xms5\" (UniqueName: \"kubernetes.io/projected/439b3e86-dbdc-4f04-a400-3ad8a62118ea-kube-api-access-4xms5\") pod \"payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss\" (UID: \"439b3e86-dbdc-4f04-a400-3ad8a62118ea\") " pod="openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss"
Sep 18 19:00:10.952370 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:10.952285    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/439b3e86-dbdc-4f04-a400-3ad8a62118ea-plugins-config-volume\") pod \"payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss\" (UID: \"439b3e86-dbdc-4f04-a400-3ad8a62118ea\") " pod="openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss"
Sep 18 19:00:10.952370 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:10.952311    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p9d7\" (UniqueName: \"kubernetes.io/projected/710cec62-813c-48a4-9c39-06379f6cff96-kube-api-access-8p9d7\") pod \"payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t\" (UID: \"710cec62-813c-48a4-9c39-06379f6cff96\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t"
Sep 18 19:00:10.952453 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:10.952425    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/710cec62-813c-48a4-9c39-06379f6cff96-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t\" (UID: \"710cec62-813c-48a4-9c39-06379f6cff96\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t"
Sep 18 19:00:11.053022 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:11.052984    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/710cec62-813c-48a4-9c39-06379f6cff96-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t\" (UID: \"710cec62-813c-48a4-9c39-06379f6cff96\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t"
Sep 18 19:00:11.053180 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:11.053088    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xms5\" (UniqueName: \"kubernetes.io/projected/439b3e86-dbdc-4f04-a400-3ad8a62118ea-kube-api-access-4xms5\") pod \"payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss\" (UID: \"439b3e86-dbdc-4f04-a400-3ad8a62118ea\") " pod="openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss"
Sep 18 19:00:11.053180 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:11.053106    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/439b3e86-dbdc-4f04-a400-3ad8a62118ea-plugins-config-volume\") pod \"payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss\" (UID: \"439b3e86-dbdc-4f04-a400-3ad8a62118ea\") " pod="openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss"
Sep 18 19:00:11.053180 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:11.053130    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p9d7\" (UniqueName: \"kubernetes.io/projected/710cec62-813c-48a4-9c39-06379f6cff96-kube-api-access-8p9d7\") pod \"payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t\" (UID: \"710cec62-813c-48a4-9c39-06379f6cff96\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t"
Sep 18 19:00:11.053737 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:11.053717    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/710cec62-813c-48a4-9c39-06379f6cff96-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t\" (UID: \"710cec62-813c-48a4-9c39-06379f6cff96\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t"
Sep 18 19:00:11.053807 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:11.053788    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/439b3e86-dbdc-4f04-a400-3ad8a62118ea-plugins-config-volume\") pod \"payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss\" (UID: \"439b3e86-dbdc-4f04-a400-3ad8a62118ea\") " pod="openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss"
Sep 18 19:00:11.063579 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:11.063555    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p9d7\" (UniqueName: \"kubernetes.io/projected/710cec62-813c-48a4-9c39-06379f6cff96-kube-api-access-8p9d7\") pod \"payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t\" (UID: \"710cec62-813c-48a4-9c39-06379f6cff96\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t"
Sep 18 19:00:11.063842 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:11.063807    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xms5\" (UniqueName: \"kubernetes.io/projected/439b3e86-dbdc-4f04-a400-3ad8a62118ea-kube-api-access-4xms5\") pod \"payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss\" (UID: \"439b3e86-dbdc-4f04-a400-3ad8a62118ea\") " pod="openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss"
Sep 18 19:00:11.181371 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:11.181333    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss"
Sep 18 19:00:11.225460 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:11.225430    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t"
Sep 18 19:00:11.474783 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:11.474760    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss"]
Sep 18 19:00:11.477577 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:00:11.477547    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod439b3e86_dbdc_4f04_a400_3ad8a62118ea.slice/crio-488eeb8607fb0aeb929fd354e6e45b45be02ed297c27f2d5f958b7e42f350f43 WatchSource:0}: Error finding container 488eeb8607fb0aeb929fd354e6e45b45be02ed297c27f2d5f958b7e42f350f43: Status 404 returned error can't find the container with id 488eeb8607fb0aeb929fd354e6e45b45be02ed297c27f2d5f958b7e42f350f43
Sep 18 19:00:11.522873 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:11.522850    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t"]
Sep 18 19:00:11.524954 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:00:11.524927    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod710cec62_813c_48a4_9c39_06379f6cff96.slice/crio-db5f99e5716062da85e42a570e0662267c570f0ee65c6ba29e92e152091b498b WatchSource:0}: Error finding container db5f99e5716062da85e42a570e0662267c570f0ee65c6ba29e92e152091b498b: Status 404 returned error can't find the container with id db5f99e5716062da85e42a570e0662267c570f0ee65c6ba29e92e152091b498b
Sep 18 19:00:12.150724 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:12.150691    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss" event={"ID":"439b3e86-dbdc-4f04-a400-3ad8a62118ea","Type":"ContainerStarted","Data":"5b5bf1d8941828ff9b5ffc4089477e97f39cb6b362bcf229f2eb952f763ca5ec"}
Sep 18 19:00:12.150946 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:12.150752    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss" event={"ID":"439b3e86-dbdc-4f04-a400-3ad8a62118ea","Type":"ContainerStarted","Data":"488eeb8607fb0aeb929fd354e6e45b45be02ed297c27f2d5f958b7e42f350f43"}
Sep 18 19:00:12.150946 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:12.150785    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss"
Sep 18 19:00:12.153616 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:12.153570    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t" event={"ID":"710cec62-813c-48a4-9c39-06379f6cff96","Type":"ContainerStarted","Data":"e9b2267b030c2701df08f5234c72d4b43c06f99f19b55591136ccef02b4b874b"}
Sep 18 19:00:12.153727 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:12.153627    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t" event={"ID":"710cec62-813c-48a4-9c39-06379f6cff96","Type":"ContainerStarted","Data":"db5f99e5716062da85e42a570e0662267c570f0ee65c6ba29e92e152091b498b"}
Sep 18 19:00:12.153727 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:12.153660    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t"
Sep 18 19:00:12.177570 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:12.177494    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss" podStartSLOduration=2.177473329 podStartE2EDuration="2.177473329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:00:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:00:12.175955024 +0000 UTC m=+1248.374439228" watchObservedRunningTime="2026-09-18 19:00:12.177473329 +0000 UTC m=+1248.375957536"
Sep 18 19:00:12.205639 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:12.205586    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t" podStartSLOduration=2.205566474 podStartE2EDuration="2.205566474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:00:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:00:12.201604586 +0000 UTC m=+1248.400088813" watchObservedRunningTime="2026-09-18 19:00:12.205566474 +0000 UTC m=+1248.404050680"
Sep 18 19:00:12.798366 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:12.798330    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:13.615005 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:13.614974    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:14.169192 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:14.169163    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-351045-29829300-x9gqz" event={"ID":"0ab9b3e9-64b0-4955-a7a4-d84cf9ca6518","Type":"ContainerStarted","Data":"2cf0d03127b1ca2ff1180f6163db7b0abb07dbd7dd764d71e2528146ffacca8f"}
Sep 18 19:00:14.211520 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:14.211471    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-351045-29829300-x9gqz" podStartSLOduration=1.25972486 podStartE2EDuration="14.211457s" totalImagesPullingTime="12.95173214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:00:14.209881585 +0000 UTC m=+1250.408365810" watchObservedRunningTime="2026-09-18 19:00:14.211457 +0000 UTC m=+1250.409941204"
Sep 18 19:00:15.131448 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:15.131411    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:15.176992 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:15.176966    2558 generic.go:356] "Generic (PLEG): container finished" podID="0ab9b3e9-64b0-4955-a7a4-d84cf9ca6518" containerID="2cf0d03127b1ca2ff1180f6163db7b0abb07dbd7dd764d71e2528146ffacca8f" exitCode=0
Sep 18 19:00:15.177153 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:15.177032    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-351045-29829300-x9gqz" event={"ID":"0ab9b3e9-64b0-4955-a7a4-d84cf9ca6518","Type":"ContainerDied","Data":"2cf0d03127b1ca2ff1180f6163db7b0abb07dbd7dd764d71e2528146ffacca8f"}
Sep 18 19:00:15.940043 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:15.940012    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:16.319886 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:16.319865    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-351045-29829300-x9gqz"
Sep 18 19:00:17.188563 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:17.188538    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-351045-29829300-x9gqz"
Sep 18 19:00:17.188749 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:17.188579    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-351045-29829300-x9gqz" event={"ID":"0ab9b3e9-64b0-4955-a7a4-d84cf9ca6518","Type":"ContainerDied","Data":"56b1d72b7ca5aec67e9a33808b0f9b06dc4595a108b8d4af97806eea18d4b7a0"}
Sep 18 19:00:17.188749 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:17.188623    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56b1d72b7ca5aec67e9a33808b0f9b06dc4595a108b8d4af97806eea18d4b7a0"
Sep 18 19:00:17.260023 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:17.259983    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:17.776154 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:17.776094    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"]
Sep 18 19:00:17.974542 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:17.974510    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:18.329847 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:18.329798    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:19.293854 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:19.293799    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:20.195600 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:20.195562    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:21.023205 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:21.023170    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:22.759481 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:22.759443    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:23.164682 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:23.164599    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t"
Sep 18 19:00:23.164831 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:23.164683    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss"
Sep 18 19:00:23.786327 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:23.786290    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:24.633445 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:24.633407    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:24.810007 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:24.809967    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:25.557545 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:25.557515    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:26.490157 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:26.490121    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:27.479478 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:27.479448    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:28.309994 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:28.309962    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:29.119171 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.119138    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:29.173174 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.173135    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66"]
Sep 18 19:00:29.173459 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.173416    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66" podUID="005a4591-36ba-487d-8fbd-5c6e034ab1f0" containerName="payload-processing" containerID="cri-o://e59c63358a379be2e1003e594538dbea3b7717d40d569342fe9ca80b62da4c1f" gracePeriod=30
Sep 18 19:00:29.176713 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.176687    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c"]
Sep 18 19:00:29.176943 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.176912    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c" podUID="02e2b271-43d7-43a2-ae46-991ae54de7bb" containerName="payload-pre-processing" containerID="cri-o://284983b2e767641e30dec222eb22d1a3183abf1544b98eabe744f0b0e1f4d3eb" gracePeriod=30
Sep 18 19:00:29.237345 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.237311    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:29.341907 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.341866    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:29.464011 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.463992    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c"
Sep 18 19:00:29.467255 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.467238    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66"
Sep 18 19:00:29.494655 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.494493    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:29.544153 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.544128    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnsg4\" (UniqueName: \"kubernetes.io/projected/005a4591-36ba-487d-8fbd-5c6e034ab1f0-kube-api-access-jnsg4\") pod \"005a4591-36ba-487d-8fbd-5c6e034ab1f0\" (UID: \"005a4591-36ba-487d-8fbd-5c6e034ab1f0\") "
Sep 18 19:00:29.544305 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.544185    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/005a4591-36ba-487d-8fbd-5c6e034ab1f0-plugins-config-volume\") pod \"005a4591-36ba-487d-8fbd-5c6e034ab1f0\" (UID: \"005a4591-36ba-487d-8fbd-5c6e034ab1f0\") "
Sep 18 19:00:29.544305 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.544245    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bk7pq\" (UniqueName: \"kubernetes.io/projected/02e2b271-43d7-43a2-ae46-991ae54de7bb-kube-api-access-bk7pq\") pod \"02e2b271-43d7-43a2-ae46-991ae54de7bb\" (UID: \"02e2b271-43d7-43a2-ae46-991ae54de7bb\") "
Sep 18 19:00:29.544305 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.544270    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02e2b271-43d7-43a2-ae46-991ae54de7bb-plugins-config-volume\") pod \"02e2b271-43d7-43a2-ae46-991ae54de7bb\" (UID: \"02e2b271-43d7-43a2-ae46-991ae54de7bb\") "
Sep 18 19:00:29.544663 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.544635    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/005a4591-36ba-487d-8fbd-5c6e034ab1f0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "005a4591-36ba-487d-8fbd-5c6e034ab1f0" (UID: "005a4591-36ba-487d-8fbd-5c6e034ab1f0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:00:29.544775 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.544662    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02e2b271-43d7-43a2-ae46-991ae54de7bb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "02e2b271-43d7-43a2-ae46-991ae54de7bb" (UID: "02e2b271-43d7-43a2-ae46-991ae54de7bb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:00:29.546814 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.546784    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/005a4591-36ba-487d-8fbd-5c6e034ab1f0-kube-api-access-jnsg4" (OuterVolumeSpecName: "kube-api-access-jnsg4") pod "005a4591-36ba-487d-8fbd-5c6e034ab1f0" (UID: "005a4591-36ba-487d-8fbd-5c6e034ab1f0"). InnerVolumeSpecName "kube-api-access-jnsg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:00:29.546814 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.546880    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02e2b271-43d7-43a2-ae46-991ae54de7bb-kube-api-access-bk7pq" (OuterVolumeSpecName: "kube-api-access-bk7pq") pod "02e2b271-43d7-43a2-ae46-991ae54de7bb" (UID: "02e2b271-43d7-43a2-ae46-991ae54de7bb"). InnerVolumeSpecName "kube-api-access-bk7pq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:00:29.645865 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.645756    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bk7pq\" (UniqueName: \"kubernetes.io/projected/02e2b271-43d7-43a2-ae46-991ae54de7bb-kube-api-access-bk7pq\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:00:29.645865 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.645783    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02e2b271-43d7-43a2-ae46-991ae54de7bb-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:00:29.645865 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.645794    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnsg4\" (UniqueName: \"kubernetes.io/projected/005a4591-36ba-487d-8fbd-5c6e034ab1f0-kube-api-access-jnsg4\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:00:29.645865 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.645802    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/005a4591-36ba-487d-8fbd-5c6e034ab1f0-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:00:29.964219 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:29.964139    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:30.081885 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.081847    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:30.266680 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.266650    2558 generic.go:356] "Generic (PLEG): container finished" podID="005a4591-36ba-487d-8fbd-5c6e034ab1f0" containerID="e59c63358a379be2e1003e594538dbea3b7717d40d569342fe9ca80b62da4c1f" exitCode=0
Sep 18 19:00:30.266886 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.266690    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66"
Sep 18 19:00:30.266886 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.266745    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66" event={"ID":"005a4591-36ba-487d-8fbd-5c6e034ab1f0","Type":"ContainerDied","Data":"e59c63358a379be2e1003e594538dbea3b7717d40d569342fe9ca80b62da4c1f"}
Sep 18 19:00:30.266886 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.266809    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66" event={"ID":"005a4591-36ba-487d-8fbd-5c6e034ab1f0","Type":"ContainerDied","Data":"f46256c9948c48dda18413b0a50c51f9bd1e8be1d22b50d1a152c068fcc5737d"}
Sep 18 19:00:30.266886 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.266862    2558 scope.go:117] "RemoveContainer" containerID="e59c63358a379be2e1003e594538dbea3b7717d40d569342fe9ca80b62da4c1f"
Sep 18 19:00:30.268199 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.268091    2558 generic.go:356] "Generic (PLEG): container finished" podID="02e2b271-43d7-43a2-ae46-991ae54de7bb" containerID="284983b2e767641e30dec222eb22d1a3183abf1544b98eabe744f0b0e1f4d3eb" exitCode=0
Sep 18 19:00:30.268199 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.268141    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c"
Sep 18 19:00:30.268199 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.268181    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c" event={"ID":"02e2b271-43d7-43a2-ae46-991ae54de7bb","Type":"ContainerDied","Data":"284983b2e767641e30dec222eb22d1a3183abf1544b98eabe744f0b0e1f4d3eb"}
Sep 18 19:00:30.268397 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.268233    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c" event={"ID":"02e2b271-43d7-43a2-ae46-991ae54de7bb","Type":"ContainerDied","Data":"4c51f88462c1de305138cedb46aeca6ff09d49997010bdf04d94c416dc94bc4b"}
Sep 18 19:00:30.275937 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.275915    2558 scope.go:117] "RemoveContainer" containerID="e59c63358a379be2e1003e594538dbea3b7717d40d569342fe9ca80b62da4c1f"
Sep 18 19:00:30.276189 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:00:30.276171    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e59c63358a379be2e1003e594538dbea3b7717d40d569342fe9ca80b62da4c1f\": container with ID starting with e59c63358a379be2e1003e594538dbea3b7717d40d569342fe9ca80b62da4c1f not found: ID does not exist" containerID="e59c63358a379be2e1003e594538dbea3b7717d40d569342fe9ca80b62da4c1f"
Sep 18 19:00:30.276244 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.276197    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e59c63358a379be2e1003e594538dbea3b7717d40d569342fe9ca80b62da4c1f"} err="failed to get container status \"e59c63358a379be2e1003e594538dbea3b7717d40d569342fe9ca80b62da4c1f\": rpc error: code = NotFound desc = could not find container \"e59c63358a379be2e1003e594538dbea3b7717d40d569342fe9ca80b62da4c1f\": container with ID starting with e59c63358a379be2e1003e594538dbea3b7717d40d569342fe9ca80b62da4c1f not found: ID does not exist"
Sep 18 19:00:30.276244 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.276213    2558 scope.go:117] "RemoveContainer" containerID="284983b2e767641e30dec222eb22d1a3183abf1544b98eabe744f0b0e1f4d3eb"
Sep 18 19:00:30.284339 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.284324    2558 scope.go:117] "RemoveContainer" containerID="284983b2e767641e30dec222eb22d1a3183abf1544b98eabe744f0b0e1f4d3eb"
Sep 18 19:00:30.284583 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:00:30.284565    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"284983b2e767641e30dec222eb22d1a3183abf1544b98eabe744f0b0e1f4d3eb\": container with ID starting with 284983b2e767641e30dec222eb22d1a3183abf1544b98eabe744f0b0e1f4d3eb not found: ID does not exist" containerID="284983b2e767641e30dec222eb22d1a3183abf1544b98eabe744f0b0e1f4d3eb"
Sep 18 19:00:30.284632 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.284590    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"284983b2e767641e30dec222eb22d1a3183abf1544b98eabe744f0b0e1f4d3eb"} err="failed to get container status \"284983b2e767641e30dec222eb22d1a3183abf1544b98eabe744f0b0e1f4d3eb\": rpc error: code = NotFound desc = could not find container \"284983b2e767641e30dec222eb22d1a3183abf1544b98eabe744f0b0e1f4d3eb\": container with ID starting with 284983b2e767641e30dec222eb22d1a3183abf1544b98eabe744f0b0e1f4d3eb not found: ID does not exist"
Sep 18 19:00:30.299139 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.299117    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c"]
Sep 18 19:00:30.305437 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.305403    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-9d9f36-6f45dc89c4-69c4c"]
Sep 18 19:00:30.321501 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.321478    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66"]
Sep 18 19:00:30.328979 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.328957    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-9d9f36-7f5568cf95-8ts66"]
Sep 18 19:00:30.967565 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:30.967521    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:31.845926 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:31.845895    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="005a4591-36ba-487d-8fbd-5c6e034ab1f0" path="/var/lib/kubelet/pods/005a4591-36ba-487d-8fbd-5c6e034ab1f0/volumes"
Sep 18 19:00:31.846284 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:31.846271    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02e2b271-43d7-43a2-ae46-991ae54de7bb" path="/var/lib/kubelet/pods/02e2b271-43d7-43a2-ae46-991ae54de7bb/volumes"
Sep 18 19:00:31.943106 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:31.943079    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:32.795467 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:32.795432    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr" podUID="09416ddf-4f59-44aa-88e8-61399586938a" containerName="main" containerID="cri-o://536073195b3c78371a5d8347e00d0c8b745fdda6a76715f7b2c6c0e75a21a6d9" gracePeriod=45
Sep 18 19:00:32.989651 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:32.989622    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:33.052411 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.052350    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 19:00:33.181577 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.181543    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-tmp-dir\") pod \"09416ddf-4f59-44aa-88e8-61399586938a\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") "
Sep 18 19:00:33.181758 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.181598    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-home\") pod \"09416ddf-4f59-44aa-88e8-61399586938a\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") "
Sep 18 19:00:33.181758 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.181616    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-model-cache\") pod \"09416ddf-4f59-44aa-88e8-61399586938a\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") "
Sep 18 19:00:33.181758 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.181741    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q862q\" (UniqueName: \"kubernetes.io/projected/09416ddf-4f59-44aa-88e8-61399586938a-kube-api-access-q862q\") pod \"09416ddf-4f59-44aa-88e8-61399586938a\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") "
Sep 18 19:00:33.181957 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.181768    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09416ddf-4f59-44aa-88e8-61399586938a-tls-certs\") pod \"09416ddf-4f59-44aa-88e8-61399586938a\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") "
Sep 18 19:00:33.181957 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.181796    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-dshm\") pod \"09416ddf-4f59-44aa-88e8-61399586938a\" (UID: \"09416ddf-4f59-44aa-88e8-61399586938a\") "
Sep 18 19:00:33.181957 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.181833    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "09416ddf-4f59-44aa-88e8-61399586938a" (UID: "09416ddf-4f59-44aa-88e8-61399586938a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:00:33.181957 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.181871    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-model-cache" (OuterVolumeSpecName: "model-cache") pod "09416ddf-4f59-44aa-88e8-61399586938a" (UID: "09416ddf-4f59-44aa-88e8-61399586938a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:00:33.181957 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.181890    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-home" (OuterVolumeSpecName: "home") pod "09416ddf-4f59-44aa-88e8-61399586938a" (UID: "09416ddf-4f59-44aa-88e8-61399586938a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:00:33.182200 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.182169    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-tmp-dir\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:00:33.182200 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.182196    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-home\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:00:33.182283 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.182210    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-model-cache\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:00:33.183936 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.183910    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-dshm" (OuterVolumeSpecName: "dshm") pod "09416ddf-4f59-44aa-88e8-61399586938a" (UID: "09416ddf-4f59-44aa-88e8-61399586938a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:00:33.184051 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.183943    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09416ddf-4f59-44aa-88e8-61399586938a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "09416ddf-4f59-44aa-88e8-61399586938a" (UID: "09416ddf-4f59-44aa-88e8-61399586938a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:00:33.184119 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.184085    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09416ddf-4f59-44aa-88e8-61399586938a-kube-api-access-q862q" (OuterVolumeSpecName: "kube-api-access-q862q") pod "09416ddf-4f59-44aa-88e8-61399586938a" (UID: "09416ddf-4f59-44aa-88e8-61399586938a"). InnerVolumeSpecName "kube-api-access-q862q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:00:33.283260 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.283230    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q862q\" (UniqueName: \"kubernetes.io/projected/09416ddf-4f59-44aa-88e8-61399586938a-kube-api-access-q862q\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:00:33.283260 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.283256    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09416ddf-4f59-44aa-88e8-61399586938a-tls-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:00:33.283260 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.283267    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09416ddf-4f59-44aa-88e8-61399586938a-dshm\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:00:33.289231 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.289206    2558 generic.go:356] "Generic (PLEG): container finished" podID="09416ddf-4f59-44aa-88e8-61399586938a" containerID="536073195b3c78371a5d8347e00d0c8b745fdda6a76715f7b2c6c0e75a21a6d9" exitCode=0
Sep 18 19:00:33.289351 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.289266    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"
Sep 18 19:00:33.289351 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.289291    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr" event={"ID":"09416ddf-4f59-44aa-88e8-61399586938a","Type":"ContainerDied","Data":"536073195b3c78371a5d8347e00d0c8b745fdda6a76715f7b2c6c0e75a21a6d9"}
Sep 18 19:00:33.289351 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.289333    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr" event={"ID":"09416ddf-4f59-44aa-88e8-61399586938a","Type":"ContainerDied","Data":"652f659957e94e677f200977496e8773c3076c89335ecedb35d6b1974dcda59c"}
Sep 18 19:00:33.289351 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.289349    2558 scope.go:117] "RemoveContainer" containerID="536073195b3c78371a5d8347e00d0c8b745fdda6a76715f7b2c6c0e75a21a6d9"
Sep 18 19:00:33.299159 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.299139    2558 scope.go:117] "RemoveContainer" containerID="536073195b3c78371a5d8347e00d0c8b745fdda6a76715f7b2c6c0e75a21a6d9"
Sep 18 19:00:33.299445 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:00:33.299423    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"536073195b3c78371a5d8347e00d0c8b745fdda6a76715f7b2c6c0e75a21a6d9\": container with ID starting with 536073195b3c78371a5d8347e00d0c8b745fdda6a76715f7b2c6c0e75a21a6d9 not found: ID does not exist" containerID="536073195b3c78371a5d8347e00d0c8b745fdda6a76715f7b2c6c0e75a21a6d9"
Sep 18 19:00:33.299508 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.299454    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"536073195b3c78371a5d8347e00d0c8b745fdda6a76715f7b2c6c0e75a21a6d9"} err="failed to get container status \"536073195b3c78371a5d8347e00d0c8b745fdda6a76715f7b2c6c0e75a21a6d9\": rpc error: code = NotFound desc = could not find container \"536073195b3c78371a5d8347e00d0c8b745fdda6a76715f7b2c6c0e75a21a6d9\": container with ID starting with 536073195b3c78371a5d8347e00d0c8b745fdda6a76715f7b2c6c0e75a21a6d9 not found: ID does not exist"
Sep 18 19:00:33.322096 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.322050    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"]
Sep 18 19:00:33.331391 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.331369    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-9d9f36/distinct-w3-9d9f36-kserve-8645cf648c-6frhr"]
Sep 18 19:00:33.845833 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:33.845789    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09416ddf-4f59-44aa-88e8-61399586938a" path="/var/lib/kubelet/pods/09416ddf-4f59-44aa-88e8-61399586938a/volumes"
Sep 18 19:00:34.059973 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:34.059939    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:34.934457 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:34.934409    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:35.792384 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:35.792348    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:36.930276 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:36.930244    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:37.031191 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:37.031154    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:37.222275 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:37.222183    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:44.069605 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:44.069568    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:55.324339 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:55.324295    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:55.445369 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:55.445334    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:57.215204 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:57.215170    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:00:59.671297 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:00:59.671263    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:01:11.818045 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:01:11.818009    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:01:21.331219 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:01:21.331185    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:01:45.322624 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:01:45.322538    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:01:45.487585 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:01:45.487545    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:01:45.559450 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:01:45.559420    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:01:46.250348 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:01:46.250313    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:01:48.176804 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:01:48.176769    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:01:48.276243 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:01:48.276207    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:01:48.840650 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:01:48.840619    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:01:49.728240 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:01:49.728211    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:01:50.430175 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:01:50.430142    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:01:51.132271 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:01:51.132232    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:00.755313 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:00.755266    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss"]
Sep 18 19:02:00.755916 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:00.755571    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss" podUID="439b3e86-dbdc-4f04-a400-3ad8a62118ea" containerName="payload-processing" containerID="cri-o://5b5bf1d8941828ff9b5ffc4089477e97f39cb6b362bcf229f2eb952f763ca5ec" gracePeriod=30
Sep 18 19:02:00.766443 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:00.766410    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t"]
Sep 18 19:02:00.767148 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:00.766957    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t" podUID="710cec62-813c-48a4-9c39-06379f6cff96" containerName="payload-pre-processing" containerID="cri-o://e9b2267b030c2701df08f5234c72d4b43c06f99f19b55591136ccef02b4b874b" gracePeriod=30
Sep 18 19:02:01.053778 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.053748    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss"
Sep 18 19:02:01.057374 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.057349    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t"
Sep 18 19:02:01.146044 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.146011    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/710cec62-813c-48a4-9c39-06379f6cff96-plugins-config-volume\") pod \"710cec62-813c-48a4-9c39-06379f6cff96\" (UID: \"710cec62-813c-48a4-9c39-06379f6cff96\") "
Sep 18 19:02:01.146205 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.146078    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xms5\" (UniqueName: \"kubernetes.io/projected/439b3e86-dbdc-4f04-a400-3ad8a62118ea-kube-api-access-4xms5\") pod \"439b3e86-dbdc-4f04-a400-3ad8a62118ea\" (UID: \"439b3e86-dbdc-4f04-a400-3ad8a62118ea\") "
Sep 18 19:02:01.146205 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.146111    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/439b3e86-dbdc-4f04-a400-3ad8a62118ea-plugins-config-volume\") pod \"439b3e86-dbdc-4f04-a400-3ad8a62118ea\" (UID: \"439b3e86-dbdc-4f04-a400-3ad8a62118ea\") "
Sep 18 19:02:01.146205 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.146173    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p9d7\" (UniqueName: \"kubernetes.io/projected/710cec62-813c-48a4-9c39-06379f6cff96-kube-api-access-8p9d7\") pod \"710cec62-813c-48a4-9c39-06379f6cff96\" (UID: \"710cec62-813c-48a4-9c39-06379f6cff96\") "
Sep 18 19:02:01.146468 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.146445    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/710cec62-813c-48a4-9c39-06379f6cff96-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "710cec62-813c-48a4-9c39-06379f6cff96" (UID: "710cec62-813c-48a4-9c39-06379f6cff96"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:02:01.146569 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.146532    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/439b3e86-dbdc-4f04-a400-3ad8a62118ea-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "439b3e86-dbdc-4f04-a400-3ad8a62118ea" (UID: "439b3e86-dbdc-4f04-a400-3ad8a62118ea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:02:01.146656 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.146595    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/710cec62-813c-48a4-9c39-06379f6cff96-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:02:01.148471 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.148429    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/710cec62-813c-48a4-9c39-06379f6cff96-kube-api-access-8p9d7" (OuterVolumeSpecName: "kube-api-access-8p9d7") pod "710cec62-813c-48a4-9c39-06379f6cff96" (UID: "710cec62-813c-48a4-9c39-06379f6cff96"). InnerVolumeSpecName "kube-api-access-8p9d7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:02:01.148610 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.148531    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/439b3e86-dbdc-4f04-a400-3ad8a62118ea-kube-api-access-4xms5" (OuterVolumeSpecName: "kube-api-access-4xms5") pod "439b3e86-dbdc-4f04-a400-3ad8a62118ea" (UID: "439b3e86-dbdc-4f04-a400-3ad8a62118ea"). InnerVolumeSpecName "kube-api-access-4xms5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:02:01.247596 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.247544    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/439b3e86-dbdc-4f04-a400-3ad8a62118ea-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:02:01.247596 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.247587    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p9d7\" (UniqueName: \"kubernetes.io/projected/710cec62-813c-48a4-9c39-06379f6cff96-kube-api-access-8p9d7\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:02:01.247596 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.247597    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xms5\" (UniqueName: \"kubernetes.io/projected/439b3e86-dbdc-4f04-a400-3ad8a62118ea-kube-api-access-4xms5\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:02:01.539127 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.539093    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:01.639516 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.639481    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:01.785966 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.785936    2558 generic.go:356] "Generic (PLEG): container finished" podID="439b3e86-dbdc-4f04-a400-3ad8a62118ea" containerID="5b5bf1d8941828ff9b5ffc4089477e97f39cb6b362bcf229f2eb952f763ca5ec" exitCode=0
Sep 18 19:02:01.786434 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.785992    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss"
Sep 18 19:02:01.786434 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.786031    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss" event={"ID":"439b3e86-dbdc-4f04-a400-3ad8a62118ea","Type":"ContainerDied","Data":"5b5bf1d8941828ff9b5ffc4089477e97f39cb6b362bcf229f2eb952f763ca5ec"}
Sep 18 19:02:01.786434 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.786085    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss" event={"ID":"439b3e86-dbdc-4f04-a400-3ad8a62118ea","Type":"ContainerDied","Data":"488eeb8607fb0aeb929fd354e6e45b45be02ed297c27f2d5f958b7e42f350f43"}
Sep 18 19:02:01.786434 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.786105    2558 scope.go:117] "RemoveContainer" containerID="5b5bf1d8941828ff9b5ffc4089477e97f39cb6b362bcf229f2eb952f763ca5ec"
Sep 18 19:02:01.787574 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.787489    2558 generic.go:356] "Generic (PLEG): container finished" podID="710cec62-813c-48a4-9c39-06379f6cff96" containerID="e9b2267b030c2701df08f5234c72d4b43c06f99f19b55591136ccef02b4b874b" exitCode=0
Sep 18 19:02:01.787574 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.787521    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t" event={"ID":"710cec62-813c-48a4-9c39-06379f6cff96","Type":"ContainerDied","Data":"e9b2267b030c2701df08f5234c72d4b43c06f99f19b55591136ccef02b4b874b"}
Sep 18 19:02:01.787681 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.787525    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t"
Sep 18 19:02:01.787681 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.787572    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t" event={"ID":"710cec62-813c-48a4-9c39-06379f6cff96","Type":"ContainerDied","Data":"db5f99e5716062da85e42a570e0662267c570f0ee65c6ba29e92e152091b498b"}
Sep 18 19:02:01.797475 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.797345    2558 scope.go:117] "RemoveContainer" containerID="5b5bf1d8941828ff9b5ffc4089477e97f39cb6b362bcf229f2eb952f763ca5ec"
Sep 18 19:02:01.797719 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:02:01.797687    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b5bf1d8941828ff9b5ffc4089477e97f39cb6b362bcf229f2eb952f763ca5ec\": container with ID starting with 5b5bf1d8941828ff9b5ffc4089477e97f39cb6b362bcf229f2eb952f763ca5ec not found: ID does not exist" containerID="5b5bf1d8941828ff9b5ffc4089477e97f39cb6b362bcf229f2eb952f763ca5ec"
Sep 18 19:02:01.797925 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.797727    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b5bf1d8941828ff9b5ffc4089477e97f39cb6b362bcf229f2eb952f763ca5ec"} err="failed to get container status \"5b5bf1d8941828ff9b5ffc4089477e97f39cb6b362bcf229f2eb952f763ca5ec\": rpc error: code = NotFound desc = could not find container \"5b5bf1d8941828ff9b5ffc4089477e97f39cb6b362bcf229f2eb952f763ca5ec\": container with ID starting with 5b5bf1d8941828ff9b5ffc4089477e97f39cb6b362bcf229f2eb952f763ca5ec not found: ID does not exist"
Sep 18 19:02:01.797925 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.797750    2558 scope.go:117] "RemoveContainer" containerID="e9b2267b030c2701df08f5234c72d4b43c06f99f19b55591136ccef02b4b874b"
Sep 18 19:02:01.806878 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.806861    2558 scope.go:117] "RemoveContainer" containerID="e9b2267b030c2701df08f5234c72d4b43c06f99f19b55591136ccef02b4b874b"
Sep 18 19:02:01.807127 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:02:01.807109    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9b2267b030c2701df08f5234c72d4b43c06f99f19b55591136ccef02b4b874b\": container with ID starting with e9b2267b030c2701df08f5234c72d4b43c06f99f19b55591136ccef02b4b874b not found: ID does not exist" containerID="e9b2267b030c2701df08f5234c72d4b43c06f99f19b55591136ccef02b4b874b"
Sep 18 19:02:01.807181 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.807133    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9b2267b030c2701df08f5234c72d4b43c06f99f19b55591136ccef02b4b874b"} err="failed to get container status \"e9b2267b030c2701df08f5234c72d4b43c06f99f19b55591136ccef02b4b874b\": rpc error: code = NotFound desc = could not find container \"e9b2267b030c2701df08f5234c72d4b43c06f99f19b55591136ccef02b4b874b\": container with ID starting with e9b2267b030c2701df08f5234c72d4b43c06f99f19b55591136ccef02b4b874b not found: ID does not exist"
Sep 18 19:02:01.808674 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.808655    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss"]
Sep 18 19:02:01.812127 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.812107    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-25dc6358-7b4d74bb97-w72ss"]
Sep 18 19:02:01.824540 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.824515    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t"]
Sep 18 19:02:01.829083 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.829059    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-25dc6358-b5c854bcd-2r74t"]
Sep 18 19:02:01.845804 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.845779    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="439b3e86-dbdc-4f04-a400-3ad8a62118ea" path="/var/lib/kubelet/pods/439b3e86-dbdc-4f04-a400-3ad8a62118ea/volumes"
Sep 18 19:02:01.846236 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:01.846219    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="710cec62-813c-48a4-9c39-06379f6cff96" path="/var/lib/kubelet/pods/710cec62-813c-48a4-9c39-06379f6cff96/volumes"
Sep 18 19:02:02.078508 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:02.078418    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:02.769860 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:02.769831    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:03.646973 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:03.646935    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:03.929619 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:03.929516    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:04.376152 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:04.376111    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:05.093128 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:05.093092    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:08.957335 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:08.957297    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:14.869216 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:14.869178    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:15.203987 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:15.203889    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:16.372609 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:16.372574    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:16.712247 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:16.712164    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:17.433560 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:17.433519    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:18.134242 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:18.134203    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:19.046447 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:19.046407    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:19.741681 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:19.741646    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:19.937262 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:19.937233    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:25.375014 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:25.374974    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:30.573318 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.573286    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"]
Sep 18 19:02:30.573793 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.573505    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl" podUID="1aee097c-ece3-4f75-b8bf-e4043bc9ce20" containerName="maas-api" containerID="cri-o://676e4d773d1d12426c51fc767f680e3f1a202a2431e752bc76d18c2937f29bb9" gracePeriod=30
Sep 18 19:02:30.605548 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.605507    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:30.641586 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.641557    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr"]
Sep 18 19:02:30.641833 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.641796    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr" podUID="a35a8ead-e3e9-4ce4-a68d-aa78e257730c" containerName="payload-processing" containerID="cri-o://266e7d2b65d8d2fc12e4e3be6cc4d60b94ef4d9246c3df15a042d07469dccd02" gracePeriod=30
Sep 18 19:02:30.649468 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.649434    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt"]
Sep 18 19:02:30.650115 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.649739    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt" podUID="ada18349-7cde-43b4-b732-9ec5ac8f0235" containerName="payload-pre-processing" containerID="cri-o://5df803e15bf9adfd6035c83ead6395038a8ae476e36eb7339a41e9fc9cead4f6" gracePeriod=30
Sep 18 19:02:30.682647 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.682622    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:30.744215 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.744189    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:30.840380 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.840296    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:30.951278 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.950947    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr"
Sep 18 19:02:30.955151 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.954938    2558 generic.go:356] "Generic (PLEG): container finished" podID="ada18349-7cde-43b4-b732-9ec5ac8f0235" containerID="5df803e15bf9adfd6035c83ead6395038a8ae476e36eb7339a41e9fc9cead4f6" exitCode=0
Sep 18 19:02:30.955151 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.955034    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt" event={"ID":"ada18349-7cde-43b4-b732-9ec5ac8f0235","Type":"ContainerDied","Data":"5df803e15bf9adfd6035c83ead6395038a8ae476e36eb7339a41e9fc9cead4f6"}
Sep 18 19:02:30.955151 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.955084    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt" event={"ID":"ada18349-7cde-43b4-b732-9ec5ac8f0235","Type":"ContainerDied","Data":"0a2a6ee9d00a34e39ee725776e6b8b47308d186d1ee842fe9a04111970d76496"}
Sep 18 19:02:30.955151 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.955108    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a2a6ee9d00a34e39ee725776e6b8b47308d186d1ee842fe9a04111970d76496"
Sep 18 19:02:30.956804 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.956779    2558 generic.go:356] "Generic (PLEG): container finished" podID="a35a8ead-e3e9-4ce4-a68d-aa78e257730c" containerID="266e7d2b65d8d2fc12e4e3be6cc4d60b94ef4d9246c3df15a042d07469dccd02" exitCode=0
Sep 18 19:02:30.956958 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.956939    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr" event={"ID":"a35a8ead-e3e9-4ce4-a68d-aa78e257730c","Type":"ContainerDied","Data":"266e7d2b65d8d2fc12e4e3be6cc4d60b94ef4d9246c3df15a042d07469dccd02"}
Sep 18 19:02:30.957030 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.956991    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr" event={"ID":"a35a8ead-e3e9-4ce4-a68d-aa78e257730c","Type":"ContainerDied","Data":"d9500a86d888ca075d3e66f789b47a7494fcdbe45ffbd5f66f96e47319ba8eca"}
Sep 18 19:02:30.957030 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.957020    2558 scope.go:117] "RemoveContainer" containerID="266e7d2b65d8d2fc12e4e3be6cc4d60b94ef4d9246c3df15a042d07469dccd02"
Sep 18 19:02:30.957202 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.957186    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr"
Sep 18 19:02:30.957474 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.957346    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt"
Sep 18 19:02:30.958902 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.958877    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:30.959681 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.959656    2558 generic.go:356] "Generic (PLEG): container finished" podID="1aee097c-ece3-4f75-b8bf-e4043bc9ce20" containerID="676e4d773d1d12426c51fc767f680e3f1a202a2431e752bc76d18c2937f29bb9" exitCode=0
Sep 18 19:02:30.959798 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.959692    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl" event={"ID":"1aee097c-ece3-4f75-b8bf-e4043bc9ce20","Type":"ContainerDied","Data":"676e4d773d1d12426c51fc767f680e3f1a202a2431e752bc76d18c2937f29bb9"}
Sep 18 19:02:30.967689 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.967670    2558 scope.go:117] "RemoveContainer" containerID="266e7d2b65d8d2fc12e4e3be6cc4d60b94ef4d9246c3df15a042d07469dccd02"
Sep 18 19:02:30.968049 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:02:30.968021    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"266e7d2b65d8d2fc12e4e3be6cc4d60b94ef4d9246c3df15a042d07469dccd02\": container with ID starting with 266e7d2b65d8d2fc12e4e3be6cc4d60b94ef4d9246c3df15a042d07469dccd02 not found: ID does not exist" containerID="266e7d2b65d8d2fc12e4e3be6cc4d60b94ef4d9246c3df15a042d07469dccd02"
Sep 18 19:02:30.968136 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:30.968055    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"266e7d2b65d8d2fc12e4e3be6cc4d60b94ef4d9246c3df15a042d07469dccd02"} err="failed to get container status \"266e7d2b65d8d2fc12e4e3be6cc4d60b94ef4d9246c3df15a042d07469dccd02\": rpc error: code = NotFound desc = could not find container \"266e7d2b65d8d2fc12e4e3be6cc4d60b94ef4d9246c3df15a042d07469dccd02\": container with ID starting with 266e7d2b65d8d2fc12e4e3be6cc4d60b94ef4d9246c3df15a042d07469dccd02 not found: ID does not exist"
Sep 18 19:02:31.035437 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.035415    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 19:02:31.037551 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.037533    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29nkf\" (UniqueName: \"kubernetes.io/projected/a35a8ead-e3e9-4ce4-a68d-aa78e257730c-kube-api-access-29nkf\") pod \"a35a8ead-e3e9-4ce4-a68d-aa78e257730c\" (UID: \"a35a8ead-e3e9-4ce4-a68d-aa78e257730c\") "
Sep 18 19:02:31.037641 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.037597    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a35a8ead-e3e9-4ce4-a68d-aa78e257730c-plugins-config-volume\") pod \"a35a8ead-e3e9-4ce4-a68d-aa78e257730c\" (UID: \"a35a8ead-e3e9-4ce4-a68d-aa78e257730c\") "
Sep 18 19:02:31.038011 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.037991    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a35a8ead-e3e9-4ce4-a68d-aa78e257730c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a35a8ead-e3e9-4ce4-a68d-aa78e257730c" (UID: "a35a8ead-e3e9-4ce4-a68d-aa78e257730c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:02:31.039560 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.039540    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a35a8ead-e3e9-4ce4-a68d-aa78e257730c-kube-api-access-29nkf" (OuterVolumeSpecName: "kube-api-access-29nkf") pod "a35a8ead-e3e9-4ce4-a68d-aa78e257730c" (UID: "a35a8ead-e3e9-4ce4-a68d-aa78e257730c"). InnerVolumeSpecName "kube-api-access-29nkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:02:31.138872 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.138779    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-metrics-certs\") pod \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\" (UID: \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\") "
Sep 18 19:02:31.138872 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.138812    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpczf\" (UniqueName: \"kubernetes.io/projected/ada18349-7cde-43b4-b732-9ec5ac8f0235-kube-api-access-hpczf\") pod \"ada18349-7cde-43b4-b732-9ec5ac8f0235\" (UID: \"ada18349-7cde-43b4-b732-9ec5ac8f0235\") "
Sep 18 19:02:31.138872 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.138856    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-openshift-service-ca\") pod \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\" (UID: \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\") "
Sep 18 19:02:31.139176 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.138886    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ada18349-7cde-43b4-b732-9ec5ac8f0235-plugins-config-volume\") pod \"ada18349-7cde-43b4-b732-9ec5ac8f0235\" (UID: \"ada18349-7cde-43b4-b732-9ec5ac8f0235\") "
Sep 18 19:02:31.139176 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.138906    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-maas-api-tls\") pod \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\" (UID: \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\") "
Sep 18 19:02:31.139176 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.138939    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rmjb\" (UniqueName: \"kubernetes.io/projected/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-kube-api-access-4rmjb\") pod \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\" (UID: \"1aee097c-ece3-4f75-b8bf-e4043bc9ce20\") "
Sep 18 19:02:31.139396 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.139300    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1aee097c-ece3-4f75-b8bf-e4043bc9ce20" (UID: "1aee097c-ece3-4f75-b8bf-e4043bc9ce20"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:02:31.139396 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.139317    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ada18349-7cde-43b4-b732-9ec5ac8f0235-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ada18349-7cde-43b4-b732-9ec5ac8f0235" (UID: "ada18349-7cde-43b4-b732-9ec5ac8f0235"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:02:31.139490 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.139450    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-openshift-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:02:31.139490 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.139468    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ada18349-7cde-43b4-b732-9ec5ac8f0235-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:02:31.139490 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.139482    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29nkf\" (UniqueName: \"kubernetes.io/projected/a35a8ead-e3e9-4ce4-a68d-aa78e257730c-kube-api-access-29nkf\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:02:31.139599 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.139496    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a35a8ead-e3e9-4ce4-a68d-aa78e257730c-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:02:31.141115 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.141092    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1aee097c-ece3-4f75-b8bf-e4043bc9ce20" (UID: "1aee097c-ece3-4f75-b8bf-e4043bc9ce20"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:02:31.141225 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.141148    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-kube-api-access-4rmjb" (OuterVolumeSpecName: "kube-api-access-4rmjb") pod "1aee097c-ece3-4f75-b8bf-e4043bc9ce20" (UID: "1aee097c-ece3-4f75-b8bf-e4043bc9ce20"). InnerVolumeSpecName "kube-api-access-4rmjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:02:31.141384 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.141361    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1aee097c-ece3-4f75-b8bf-e4043bc9ce20" (UID: "1aee097c-ece3-4f75-b8bf-e4043bc9ce20"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:02:31.141384 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.141365    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ada18349-7cde-43b4-b732-9ec5ac8f0235-kube-api-access-hpczf" (OuterVolumeSpecName: "kube-api-access-hpczf") pod "ada18349-7cde-43b4-b732-9ec5ac8f0235" (UID: "ada18349-7cde-43b4-b732-9ec5ac8f0235"). InnerVolumeSpecName "kube-api-access-hpczf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:02:31.195906 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.195875    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:31.240996 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.240970    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-maas-api-tls\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:02:31.240996 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.240993    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rmjb\" (UniqueName: \"kubernetes.io/projected/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-kube-api-access-4rmjb\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:02:31.241157 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.241003    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1aee097c-ece3-4f75-b8bf-e4043bc9ce20-metrics-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:02:31.241157 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.241013    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpczf\" (UniqueName: \"kubernetes.io/projected/ada18349-7cde-43b4-b732-9ec5ac8f0235-kube-api-access-hpczf\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:02:31.281093 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.281062    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr"]
Sep 18 19:02:31.288498 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.288474    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-cc289c-8554c969bc-5q9nr"]
Sep 18 19:02:31.309812 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.309786    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:31.846748 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.846717    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a35a8ead-e3e9-4ce4-a68d-aa78e257730c" path="/var/lib/kubelet/pods/a35a8ead-e3e9-4ce4-a68d-aa78e257730c/volumes"
Sep 18 19:02:31.873442 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.873407    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:31.966467 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.966392    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"
Sep 18 19:02:31.966467 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.966451    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl" event={"ID":"1aee097c-ece3-4f75-b8bf-e4043bc9ce20","Type":"ContainerDied","Data":"0f71df8172d06607fee9fb53854fcabdc1110133532de9a38c62655c9db6d04f"}
Sep 18 19:02:31.966663 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.966514    2558 scope.go:117] "RemoveContainer" containerID="676e4d773d1d12426c51fc767f680e3f1a202a2431e752bc76d18c2937f29bb9"
Sep 18 19:02:31.967929 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.967531    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt"
Sep 18 19:02:31.986035 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.986011    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"]
Sep 18 19:02:31.989592 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:31.989569    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc289c-57bc58d84b-qj2kl"]
Sep 18 19:02:32.000676 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:32.000655    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt"]
Sep 18 19:02:32.004665 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:32.004636    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-cc289c-579fb59b6b-9h7pt"]
Sep 18 19:02:32.547073 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:32.547039    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:33.448428 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:33.448389    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:33.845250 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:33.845223    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1aee097c-ece3-4f75-b8bf-e4043bc9ce20" path="/var/lib/kubelet/pods/1aee097c-ece3-4f75-b8bf-e4043bc9ce20/volumes"
Sep 18 19:02:33.845579 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:33.845565    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ada18349-7cde-43b4-b732-9ec5ac8f0235" path="/var/lib/kubelet/pods/ada18349-7cde-43b4-b732-9ec5ac8f0235/volumes"
Sep 18 19:02:34.174873 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:34.174768    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:37.936465 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:37.936429    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:38.036624 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:38.036596    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:38.168655 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:38.168624    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:41.452442 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:41.452412    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:41.507963 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:41.507934    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:44.385705 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:44.385550    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:44.439675 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:44.439608    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:45.198546 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:45.198507    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:46.016146 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:46.016108    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:48.444649 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:48.444617    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:50.350842 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:50.350790    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:51.612289 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:51.612248    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:51.713907 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:51.713876    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:51.853212 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:51.853180    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:53.049385 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:53.049353    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:53.153855 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:53.153800    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:53.248409 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:53.248375    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:53.817496 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:53.817462    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:53.913682 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:53.913648    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:54.013264 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:54.013236    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:55.267302 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:55.267269    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:56.048139 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:56.048093    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:56.148363 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:56.148325    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:02:57.682572 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:02:57.682532    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:01.509229 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:01.509192    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:01.601168 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:01.601133    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:02.191658 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:02.191626    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:03.573655 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:03.573615    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:08.003462 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:08.003371    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:08.107503 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:08.107448    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:08.208540 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:08.208507    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:18.646246 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:18.646207    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:18.848564 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:18.848524    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:25.981937 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:25.981904    2558 scope.go:117] "RemoveContainer" containerID="5df803e15bf9adfd6035c83ead6395038a8ae476e36eb7339a41e9fc9cead4f6"
Sep 18 19:03:39.738276 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:39.738241    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:45.640667 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:45.640635    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:45.740580 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:45.740545    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:45.976076 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:45.975993    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:46.266509 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:46.266470    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:46.371138 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:46.371105    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:48.926808 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:48.926773    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"]
Sep 18 19:03:48.945758 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:48.945724    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:49.333403 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:49.333374    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:49.660925 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:49.660810    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:49.775783 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:49.775740    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:50.247058 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:50.247024    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:50.343042 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:50.343011    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:51.136398 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:51.136362    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:51.808966 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:51.808931    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:52.544912 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.544879    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"]
Sep 18 19:03:52.545566 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545543    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a35a8ead-e3e9-4ce4-a68d-aa78e257730c" containerName="payload-processing"
Sep 18 19:03:52.545670 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545570    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a35a8ead-e3e9-4ce4-a68d-aa78e257730c" containerName="payload-processing"
Sep 18 19:03:52.545670 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545590    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02e2b271-43d7-43a2-ae46-991ae54de7bb" containerName="payload-pre-processing"
Sep 18 19:03:52.545670 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545601    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="02e2b271-43d7-43a2-ae46-991ae54de7bb" containerName="payload-pre-processing"
Sep 18 19:03:52.545670 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545612    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="710cec62-813c-48a4-9c39-06379f6cff96" containerName="payload-pre-processing"
Sep 18 19:03:52.545670 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545621    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="710cec62-813c-48a4-9c39-06379f6cff96" containerName="payload-pre-processing"
Sep 18 19:03:52.545670 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545641    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="439b3e86-dbdc-4f04-a400-3ad8a62118ea" containerName="payload-processing"
Sep 18 19:03:52.545670 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545650    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="439b3e86-dbdc-4f04-a400-3ad8a62118ea" containerName="payload-processing"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545677    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ada18349-7cde-43b4-b732-9ec5ac8f0235" containerName="payload-pre-processing"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545686    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ada18349-7cde-43b4-b732-9ec5ac8f0235" containerName="payload-pre-processing"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545696    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09416ddf-4f59-44aa-88e8-61399586938a" containerName="main"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545705    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="09416ddf-4f59-44aa-88e8-61399586938a" containerName="main"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545721    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aee097c-ece3-4f75-b8bf-e4043bc9ce20" containerName="maas-api"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545730    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aee097c-ece3-4f75-b8bf-e4043bc9ce20" containerName="maas-api"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545741    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="005a4591-36ba-487d-8fbd-5c6e034ab1f0" containerName="payload-processing"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545749    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="005a4591-36ba-487d-8fbd-5c6e034ab1f0" containerName="payload-processing"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545758    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ab9b3e9-64b0-4955-a7a4-d84cf9ca6518" containerName="cleanup"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545767    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ab9b3e9-64b0-4955-a7a4-d84cf9ca6518" containerName="cleanup"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545878    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a35a8ead-e3e9-4ce4-a68d-aa78e257730c" containerName="payload-processing"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545887    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ada18349-7cde-43b4-b732-9ec5ac8f0235" containerName="payload-pre-processing"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545893    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="09416ddf-4f59-44aa-88e8-61399586938a" containerName="main"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545900    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="005a4591-36ba-487d-8fbd-5c6e034ab1f0" containerName="payload-processing"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545907    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="710cec62-813c-48a4-9c39-06379f6cff96" containerName="payload-pre-processing"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545916    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="02e2b271-43d7-43a2-ae46-991ae54de7bb" containerName="payload-pre-processing"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545922    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aee097c-ece3-4f75-b8bf-e4043bc9ce20" containerName="maas-api"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545927    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ab9b3e9-64b0-4955-a7a4-d84cf9ca6518" containerName="cleanup"
Sep 18 19:03:52.545961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.545933    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="439b3e86-dbdc-4f04-a400-3ad8a62118ea" containerName="payload-processing"
Sep 18 19:03:52.549213 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.549194    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"
Sep 18 19:03:52.551708 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.551687    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-ce3141-serving-cert\""
Sep 18 19:03:52.558483 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.557805    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"]
Sep 18 19:03:52.576837 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.576785    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:52.676057 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.676026    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:52.694361 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.694333    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b49feb8e-662b-4471-99cc-8ae5dc78438a-maas-api-tls\") pod \"maas-api-e2e-api-a-ce3141-68c857f595-8zxc5\" (UID: \"b49feb8e-662b-4471-99cc-8ae5dc78438a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"
Sep 18 19:03:52.694475 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.694388    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b49feb8e-662b-4471-99cc-8ae5dc78438a-metrics-certs\") pod \"maas-api-e2e-api-a-ce3141-68c857f595-8zxc5\" (UID: \"b49feb8e-662b-4471-99cc-8ae5dc78438a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"
Sep 18 19:03:52.694475 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.694418    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqbgh\" (UniqueName: \"kubernetes.io/projected/b49feb8e-662b-4471-99cc-8ae5dc78438a-kube-api-access-bqbgh\") pod \"maas-api-e2e-api-a-ce3141-68c857f595-8zxc5\" (UID: \"b49feb8e-662b-4471-99cc-8ae5dc78438a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"
Sep 18 19:03:52.694475 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.694462    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b49feb8e-662b-4471-99cc-8ae5dc78438a-openshift-service-ca\") pod \"maas-api-e2e-api-a-ce3141-68c857f595-8zxc5\" (UID: \"b49feb8e-662b-4471-99cc-8ae5dc78438a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"
Sep 18 19:03:52.784612 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.784576    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:52.795774 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.795705    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b49feb8e-662b-4471-99cc-8ae5dc78438a-openshift-service-ca\") pod \"maas-api-e2e-api-a-ce3141-68c857f595-8zxc5\" (UID: \"b49feb8e-662b-4471-99cc-8ae5dc78438a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"
Sep 18 19:03:52.795911 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.795775    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b49feb8e-662b-4471-99cc-8ae5dc78438a-maas-api-tls\") pod \"maas-api-e2e-api-a-ce3141-68c857f595-8zxc5\" (UID: \"b49feb8e-662b-4471-99cc-8ae5dc78438a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"
Sep 18 19:03:52.795911 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.795848    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b49feb8e-662b-4471-99cc-8ae5dc78438a-metrics-certs\") pod \"maas-api-e2e-api-a-ce3141-68c857f595-8zxc5\" (UID: \"b49feb8e-662b-4471-99cc-8ae5dc78438a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"
Sep 18 19:03:52.795911 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.795871    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqbgh\" (UniqueName: \"kubernetes.io/projected/b49feb8e-662b-4471-99cc-8ae5dc78438a-kube-api-access-bqbgh\") pod \"maas-api-e2e-api-a-ce3141-68c857f595-8zxc5\" (UID: \"b49feb8e-662b-4471-99cc-8ae5dc78438a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"
Sep 18 19:03:52.796360 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.796341    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b49feb8e-662b-4471-99cc-8ae5dc78438a-openshift-service-ca\") pod \"maas-api-e2e-api-a-ce3141-68c857f595-8zxc5\" (UID: \"b49feb8e-662b-4471-99cc-8ae5dc78438a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"
Sep 18 19:03:52.798259 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.798238    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b49feb8e-662b-4471-99cc-8ae5dc78438a-metrics-certs\") pod \"maas-api-e2e-api-a-ce3141-68c857f595-8zxc5\" (UID: \"b49feb8e-662b-4471-99cc-8ae5dc78438a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"
Sep 18 19:03:52.798355 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.798314    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b49feb8e-662b-4471-99cc-8ae5dc78438a-maas-api-tls\") pod \"maas-api-e2e-api-a-ce3141-68c857f595-8zxc5\" (UID: \"b49feb8e-662b-4471-99cc-8ae5dc78438a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"
Sep 18 19:03:52.803439 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.803419    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqbgh\" (UniqueName: \"kubernetes.io/projected/b49feb8e-662b-4471-99cc-8ae5dc78438a-kube-api-access-bqbgh\") pod \"maas-api-e2e-api-a-ce3141-68c857f595-8zxc5\" (UID: \"b49feb8e-662b-4471-99cc-8ae5dc78438a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"
Sep 18 19:03:52.862996 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:52.862969    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"
Sep 18 19:03:53.139934 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.139906    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"]
Sep 18 19:03:53.142058 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:03:53.142031    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb49feb8e_662b_4471_99cc_8ae5dc78438a.slice/crio-1c9279a43ba874dc95934e42e99ffc8f2906c5781c5810303990b1f5cb254335 WatchSource:0}: Error finding container 1c9279a43ba874dc95934e42e99ffc8f2906c5781c5810303990b1f5cb254335: Status 404 returned error can't find the container with id 1c9279a43ba874dc95934e42e99ffc8f2906c5781c5810303990b1f5cb254335
Sep 18 19:03:53.143482 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.143464    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 19:03:53.154835 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.154758    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:53.249782 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.249742    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:03:53.254228 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.254205    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd"]
Sep 18 19:03:53.259081 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.259062    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd"
Sep 18 19:03:53.261582 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.261564    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-ce3141\""
Sep 18 19:03:53.261715 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.261700    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-ce3141-dockercfg-nvxxl\""
Sep 18 19:03:53.267784 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.267756    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd"]
Sep 18 19:03:53.283397 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.283370    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk"]
Sep 18 19:03:53.287294 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.287274    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk"
Sep 18 19:03:53.297979 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.297958    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk"]
Sep 18 19:03:53.389130 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.389051    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5" event={"ID":"b49feb8e-662b-4471-99cc-8ae5dc78438a","Type":"ContainerStarted","Data":"1c9279a43ba874dc95934e42e99ffc8f2906c5781c5810303990b1f5cb254335"}
Sep 18 19:03:53.402583 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.402554    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b573a274-0f3c-4cdc-a7ce-19cbb152ce8a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd\" (UID: \"b573a274-0f3c-4cdc-a7ce-19cbb152ce8a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd"
Sep 18 19:03:53.402748 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.402613    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f44d9f25-26a3-4cb4-885c-816a45dbbc1c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk\" (UID: \"f44d9f25-26a3-4cb4-885c-816a45dbbc1c\") " pod="openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk"
Sep 18 19:03:53.402748 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.402659    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8kt6\" (UniqueName: \"kubernetes.io/projected/b573a274-0f3c-4cdc-a7ce-19cbb152ce8a-kube-api-access-d8kt6\") pod \"payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd\" (UID: \"b573a274-0f3c-4cdc-a7ce-19cbb152ce8a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd"
Sep 18 19:03:53.402748 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.402728    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrnsz\" (UniqueName: \"kubernetes.io/projected/f44d9f25-26a3-4cb4-885c-816a45dbbc1c-kube-api-access-nrnsz\") pod \"payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk\" (UID: \"f44d9f25-26a3-4cb4-885c-816a45dbbc1c\") " pod="openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk"
Sep 18 19:03:53.504115 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.504083    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrnsz\" (UniqueName: \"kubernetes.io/projected/f44d9f25-26a3-4cb4-885c-816a45dbbc1c-kube-api-access-nrnsz\") pod \"payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk\" (UID: \"f44d9f25-26a3-4cb4-885c-816a45dbbc1c\") " pod="openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk"
Sep 18 19:03:53.504266 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.504136    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b573a274-0f3c-4cdc-a7ce-19cbb152ce8a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd\" (UID: \"b573a274-0f3c-4cdc-a7ce-19cbb152ce8a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd"
Sep 18 19:03:53.504266 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.504179    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f44d9f25-26a3-4cb4-885c-816a45dbbc1c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk\" (UID: \"f44d9f25-26a3-4cb4-885c-816a45dbbc1c\") " pod="openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk"
Sep 18 19:03:53.504266 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.504227    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8kt6\" (UniqueName: \"kubernetes.io/projected/b573a274-0f3c-4cdc-a7ce-19cbb152ce8a-kube-api-access-d8kt6\") pod \"payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd\" (UID: \"b573a274-0f3c-4cdc-a7ce-19cbb152ce8a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd"
Sep 18 19:03:53.504900 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.504882    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b573a274-0f3c-4cdc-a7ce-19cbb152ce8a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd\" (UID: \"b573a274-0f3c-4cdc-a7ce-19cbb152ce8a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd"
Sep 18 19:03:53.504967 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.504911    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f44d9f25-26a3-4cb4-885c-816a45dbbc1c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk\" (UID: \"f44d9f25-26a3-4cb4-885c-816a45dbbc1c\") " pod="openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk"
Sep 18 19:03:53.512939 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.512919    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrnsz\" (UniqueName: \"kubernetes.io/projected/f44d9f25-26a3-4cb4-885c-816a45dbbc1c-kube-api-access-nrnsz\") pod \"payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk\" (UID: \"f44d9f25-26a3-4cb4-885c-816a45dbbc1c\") " pod="openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk"
Sep 18 19:03:53.513106 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.513090    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8kt6\" (UniqueName: \"kubernetes.io/projected/b573a274-0f3c-4cdc-a7ce-19cbb152ce8a-kube-api-access-d8kt6\") pod \"payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd\" (UID: \"b573a274-0f3c-4cdc-a7ce-19cbb152ce8a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd"
Sep 18 19:03:53.571397 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.571375    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd"
Sep 18 19:03:53.597977 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.597922    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk"
Sep 18 19:03:53.847104 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.847084    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd"]
Sep 18 19:03:53.848770 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:03:53.848721    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb573a274_0f3c_4cdc_a7ce_19cbb152ce8a.slice/crio-393286f5660dd8febf9f545b779fe9f2362667650faa1f04f10604772459faf8 WatchSource:0}: Error finding container 393286f5660dd8febf9f545b779fe9f2362667650faa1f04f10604772459faf8: Status 404 returned error can't find the container with id 393286f5660dd8febf9f545b779fe9f2362667650faa1f04f10604772459faf8
Sep 18 19:03:53.875551 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:53.875530    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk"]
Sep 18 19:03:53.877492 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:03:53.877471    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf44d9f25_26a3_4cb4_885c_816a45dbbc1c.slice/crio-83c6e70efd74c708508751cb276020b4ed079ea998d0fb6dfd3b665a5236127d WatchSource:0}: Error finding container 83c6e70efd74c708508751cb276020b4ed079ea998d0fb6dfd3b665a5236127d: Status 404 returned error can't find the container with id 83c6e70efd74c708508751cb276020b4ed079ea998d0fb6dfd3b665a5236127d
Sep 18 19:03:54.396753 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:54.396717    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk" event={"ID":"f44d9f25-26a3-4cb4-885c-816a45dbbc1c","Type":"ContainerStarted","Data":"1173e1115dad77cf9f1b6db29aefae74204d064c08d4c5639f20959502f3f24a"}
Sep 18 19:03:54.396969 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:54.396807    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk" event={"ID":"f44d9f25-26a3-4cb4-885c-816a45dbbc1c","Type":"ContainerStarted","Data":"83c6e70efd74c708508751cb276020b4ed079ea998d0fb6dfd3b665a5236127d"}
Sep 18 19:03:54.396969 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:54.396851    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk"
Sep 18 19:03:54.398498 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:54.398470    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd" event={"ID":"b573a274-0f3c-4cdc-a7ce-19cbb152ce8a","Type":"ContainerStarted","Data":"db5c8974f94525ef51f50c91fa30e930f30f291438978c6eda6085363bb59569"}
Sep 18 19:03:54.398627 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:54.398519    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd" event={"ID":"b573a274-0f3c-4cdc-a7ce-19cbb152ce8a","Type":"ContainerStarted","Data":"393286f5660dd8febf9f545b779fe9f2362667650faa1f04f10604772459faf8"}
Sep 18 19:03:54.398627 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:54.398550    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd"
Sep 18 19:03:54.400131 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:54.400109    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5" event={"ID":"b49feb8e-662b-4471-99cc-8ae5dc78438a","Type":"ContainerStarted","Data":"fe207efd0d200729cc4edd3641a716da23f081bf8a1e6c37ce6612766465c2e4"}
Sep 18 19:03:54.400242 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:54.400229    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"
Sep 18 19:03:54.413081 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:54.413032    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk" podStartSLOduration=1.413016279 podStartE2EDuration="1.413016279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:03:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:03:54.411478808 +0000 UTC m=+1470.609963029" watchObservedRunningTime="2026-09-18 19:03:54.413016279 +0000 UTC m=+1470.611500487"
Sep 18 19:03:54.430900 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:54.430859    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5" podStartSLOduration=2.061712947 podStartE2EDuration="2.430847643s" totalImagesPullingTime="369.134696ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:03:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:03:54.428602927 +0000 UTC m=+1470.627087151" watchObservedRunningTime="2026-09-18 19:03:54.430847643 +0000 UTC m=+1470.629331847"
Sep 18 19:03:54.445638 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:03:54.445592    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd" podStartSLOduration=1.4455746409999999 podStartE2EDuration="1.445574641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:03:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:03:54.444153664 +0000 UTC m=+1470.642637870" watchObservedRunningTime="2026-09-18 19:03:54.445574641 +0000 UTC m=+1470.644058848"
Sep 18 19:04:00.410506 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:00.410479    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"
Sep 18 19:04:03.947137 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:03.947076    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2" podUID="b5de8fae-16b2-4b47-9941-9d436dcc2efd" containerName="main" containerID="cri-o://0e7fdf0307c04f5c1e61b29f51ba1d9e02bb4161bb50e6b7763754cfa6f1e3cf" gracePeriod=45
Sep 18 19:04:04.104415 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.104358    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2" podUID="b5de8fae-16b2-4b47-9941-9d436dcc2efd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": dial tcp 10.134.0.67:8000: connect: connection refused"
Sep 18 19:04:04.190720 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.190697    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 19:04:04.209338 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.209273    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-home\") pod \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") "
Sep 18 19:04:04.209338 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.209327    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-tmp-dir\") pod \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") "
Sep 18 19:04:04.209537 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.209413    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-model-cache\") pod \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") "
Sep 18 19:04:04.209537 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.209457    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-dshm\") pod \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") "
Sep 18 19:04:04.209537 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.209496    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5de8fae-16b2-4b47-9941-9d436dcc2efd-tls-certs\") pod \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") "
Sep 18 19:04:04.209537 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.209521    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvbxm\" (UniqueName: \"kubernetes.io/projected/b5de8fae-16b2-4b47-9941-9d436dcc2efd-kube-api-access-jvbxm\") pod \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\" (UID: \"b5de8fae-16b2-4b47-9941-9d436dcc2efd\") "
Sep 18 19:04:04.209752 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.209539    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-home" (OuterVolumeSpecName: "home") pod "b5de8fae-16b2-4b47-9941-9d436dcc2efd" (UID: "b5de8fae-16b2-4b47-9941-9d436dcc2efd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:04:04.209752 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.209645    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-model-cache" (OuterVolumeSpecName: "model-cache") pod "b5de8fae-16b2-4b47-9941-9d436dcc2efd" (UID: "b5de8fae-16b2-4b47-9941-9d436dcc2efd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:04:04.209989 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.209959    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b5de8fae-16b2-4b47-9941-9d436dcc2efd" (UID: "b5de8fae-16b2-4b47-9941-9d436dcc2efd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:04:04.210066 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.209988    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-home\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:04:04.210066 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.210008    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-model-cache\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:04:04.212415 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.212291    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-dshm" (OuterVolumeSpecName: "dshm") pod "b5de8fae-16b2-4b47-9941-9d436dcc2efd" (UID: "b5de8fae-16b2-4b47-9941-9d436dcc2efd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:04:04.212415 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.212310    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5de8fae-16b2-4b47-9941-9d436dcc2efd-kube-api-access-jvbxm" (OuterVolumeSpecName: "kube-api-access-jvbxm") pod "b5de8fae-16b2-4b47-9941-9d436dcc2efd" (UID: "b5de8fae-16b2-4b47-9941-9d436dcc2efd"). InnerVolumeSpecName "kube-api-access-jvbxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:04:04.212415 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.212357    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5de8fae-16b2-4b47-9941-9d436dcc2efd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b5de8fae-16b2-4b47-9941-9d436dcc2efd" (UID: "b5de8fae-16b2-4b47-9941-9d436dcc2efd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:04:04.310983 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.310935    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-dshm\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:04:04.310983 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.310975    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5de8fae-16b2-4b47-9941-9d436dcc2efd-tls-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:04:04.310983 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.310988    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvbxm\" (UniqueName: \"kubernetes.io/projected/b5de8fae-16b2-4b47-9941-9d436dcc2efd-kube-api-access-jvbxm\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:04:04.311218 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.311015    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5de8fae-16b2-4b47-9941-9d436dcc2efd-tmp-dir\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:04:04.455298 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.455263    2558 generic.go:356] "Generic (PLEG): container finished" podID="b5de8fae-16b2-4b47-9941-9d436dcc2efd" containerID="0e7fdf0307c04f5c1e61b29f51ba1d9e02bb4161bb50e6b7763754cfa6f1e3cf" exitCode=0
Sep 18 19:04:04.455488 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.455316    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"
Sep 18 19:04:04.455488 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.455316    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2" event={"ID":"b5de8fae-16b2-4b47-9941-9d436dcc2efd","Type":"ContainerDied","Data":"0e7fdf0307c04f5c1e61b29f51ba1d9e02bb4161bb50e6b7763754cfa6f1e3cf"}
Sep 18 19:04:04.455488 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.455435    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2" event={"ID":"b5de8fae-16b2-4b47-9941-9d436dcc2efd","Type":"ContainerDied","Data":"9dab51324a0e6892aa2bb64e0384cb654c748718a11d111afbc5d6631bcbd87e"}
Sep 18 19:04:04.455488 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.455459    2558 scope.go:117] "RemoveContainer" containerID="0e7fdf0307c04f5c1e61b29f51ba1d9e02bb4161bb50e6b7763754cfa6f1e3cf"
Sep 18 19:04:04.466021 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.465999    2558 scope.go:117] "RemoveContainer" containerID="0e7fdf0307c04f5c1e61b29f51ba1d9e02bb4161bb50e6b7763754cfa6f1e3cf"
Sep 18 19:04:04.466303 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:04:04.466279    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e7fdf0307c04f5c1e61b29f51ba1d9e02bb4161bb50e6b7763754cfa6f1e3cf\": container with ID starting with 0e7fdf0307c04f5c1e61b29f51ba1d9e02bb4161bb50e6b7763754cfa6f1e3cf not found: ID does not exist" containerID="0e7fdf0307c04f5c1e61b29f51ba1d9e02bb4161bb50e6b7763754cfa6f1e3cf"
Sep 18 19:04:04.466383 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.466312    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e7fdf0307c04f5c1e61b29f51ba1d9e02bb4161bb50e6b7763754cfa6f1e3cf"} err="failed to get container status \"0e7fdf0307c04f5c1e61b29f51ba1d9e02bb4161bb50e6b7763754cfa6f1e3cf\": rpc error: code = NotFound desc = could not find container \"0e7fdf0307c04f5c1e61b29f51ba1d9e02bb4161bb50e6b7763754cfa6f1e3cf\": container with ID starting with 0e7fdf0307c04f5c1e61b29f51ba1d9e02bb4161bb50e6b7763754cfa6f1e3cf not found: ID does not exist"
Sep 18 19:04:04.478294 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.478270    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"]
Sep 18 19:04:04.484996 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:04.484968    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-298125/auth-test-model-298125-kserve-7b5998f9c9-m4ch2"]
Sep 18 19:04:05.407518 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:05.407483    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk"
Sep 18 19:04:05.408002 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:05.407572    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd"
Sep 18 19:04:05.846772 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:05.846739    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5de8fae-16b2-4b47-9941-9d436dcc2efd" path="/var/lib/kubelet/pods/b5de8fae-16b2-4b47-9941-9d436dcc2efd/volumes"
Sep 18 19:04:08.789118 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:08.789077    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:08.859246 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:08.859215    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:13.642622 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:13.642580    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:14.773887 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:14.773832    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:15.468192 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:15.468155    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:16.140020 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:16.139983    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:16.813718 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:16.813677    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:17.510962 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:17.510927    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:18.258146 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:18.258107    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:19.406250 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:19.406214    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:19.663414 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:19.663328    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:20.147467 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:20.147428    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:20.442988 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:20.442906    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:20.556262 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:20.556225    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:21.878534 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:21.878495    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:23.552237 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:23.552199    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:24.227257 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:24.227219    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:24.327874 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:24.327810    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:24.427377 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:24.427346    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:25.258246 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:25.258208    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:25.855792 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:25.855762    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:25.875288 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:25.875264    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bwsg_636f0979-fe2e-49fd-86d6-31bd7961e736/ovn-acl-logging/0.log"
Sep 18 19:04:25.885385 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:25.885365    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bwsg_636f0979-fe2e-49fd-86d6-31bd7961e736/ovn-acl-logging/0.log"
Sep 18 19:04:26.737115 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:26.737079    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:27.415633 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:27.415600    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:28.051668 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.051632    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"]
Sep 18 19:04:28.052244 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.052227    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5de8fae-16b2-4b47-9941-9d436dcc2efd" containerName="main"
Sep 18 19:04:28.052244 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.052246    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5de8fae-16b2-4b47-9941-9d436dcc2efd" containerName="main"
Sep 18 19:04:28.052346 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.052318    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5de8fae-16b2-4b47-9941-9d436dcc2efd" containerName="main"
Sep 18 19:04:28.056850 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.056807    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:04:28.059426 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.059341    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-02e2c9-serving-cert\""
Sep 18 19:04:28.066976 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.066954    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"]
Sep 18 19:04:28.090687 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.090651    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:28.143386 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.143351    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-openshift-service-ca\") pod \"maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5\" (UID: \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:04:28.143545 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.143411    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-metrics-certs\") pod \"maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5\" (UID: \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:04:28.143545 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.143494    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4k9m\" (UniqueName: \"kubernetes.io/projected/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-kube-api-access-k4k9m\") pod \"maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5\" (UID: \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:04:28.143621 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.143587    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-maas-api-tls\") pod \"maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5\" (UID: \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:04:28.191557 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.191523    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:28.244373 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.244338    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-openshift-service-ca\") pod \"maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5\" (UID: \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:04:28.244524 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.244393    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-metrics-certs\") pod \"maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5\" (UID: \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:04:28.244524 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.244419    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4k9m\" (UniqueName: \"kubernetes.io/projected/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-kube-api-access-k4k9m\") pod \"maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5\" (UID: \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:04:28.244524 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.244459    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-maas-api-tls\") pod \"maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5\" (UID: \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:04:28.244661 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:04:28.244562    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-serving-cert: secret "maas-api-e2e-api-b-02e2c9-serving-cert" not found
Sep 18 19:04:28.244661 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:04:28.244615    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-maas-api-tls podName:4c0ebd9e-54bb-46a2-b81c-8f331512e0d6 nodeName:}" failed. No retries permitted until 2026-09-18 19:04:28.744599904 +0000 UTC m=+1504.943084085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-maas-api-tls") pod "maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5" (UID: "4c0ebd9e-54bb-46a2-b81c-8f331512e0d6") : secret "maas-api-e2e-api-b-02e2c9-serving-cert" not found
Sep 18 19:04:28.245078 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.245056    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-openshift-service-ca\") pod \"maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5\" (UID: \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:04:28.246918 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.246897    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-metrics-certs\") pod \"maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5\" (UID: \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:04:28.255341 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.255319    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4k9m\" (UniqueName: \"kubernetes.io/projected/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-kube-api-access-k4k9m\") pod \"maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5\" (UID: \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:04:28.562584 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.562551    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:28.659257 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.659222    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:28.736393 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.736351    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:28.750960 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.750934    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-maas-api-tls\") pod \"maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5\" (UID: \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:04:28.753259 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.753239    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-maas-api-tls\") pod \"maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5\" (UID: \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:04:28.969583 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:28.969497    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:04:29.105835 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:29.105789    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:29.244003 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:29.243973    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"]
Sep 18 19:04:29.245653 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:04:29.245627    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c0ebd9e_54bb_46a2_b81c_8f331512e0d6.slice/crio-0e91aeb4a6ff2aa0311f2b252890512e4e7e3ff85e3407191b6162bf187b4418 WatchSource:0}: Error finding container 0e91aeb4a6ff2aa0311f2b252890512e4e7e3ff85e3407191b6162bf187b4418: Status 404 returned error can't find the container with id 0e91aeb4a6ff2aa0311f2b252890512e4e7e3ff85e3407191b6162bf187b4418
Sep 18 19:04:29.597623 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:29.597544    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5" event={"ID":"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6","Type":"ContainerStarted","Data":"0e91aeb4a6ff2aa0311f2b252890512e4e7e3ff85e3407191b6162bf187b4418"}
Sep 18 19:04:29.899474 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:29.899380    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:30.604623 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:30.604590    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5" event={"ID":"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6","Type":"ContainerStarted","Data":"2d5797cffa583180cd903c3829c83452129bbbbe36a622ffd3800a517a025a7d"}
Sep 18 19:04:30.605028 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:30.604648    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:04:30.624918 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:30.624872    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5" podStartSLOduration=2.094130336 podStartE2EDuration="2.624857866s" totalImagesPullingTime="530.72753ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:04:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:04:30.622912614 +0000 UTC m=+1506.821396837" watchObservedRunningTime="2026-09-18 19:04:30.624857866 +0000 UTC m=+1506.823342070"
Sep 18 19:04:36.616721 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:36.616691    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:04:38.164708 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:38.164674    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:50.348222 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:50.348183    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:51.132783 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:51.132743    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:51.725869 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:51.725834    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:52.368429 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:52.368395    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:53.240486 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:53.240447    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:54.657611 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:54.657569    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:57.535688 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:57.535653    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:58.248974 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:58.248942    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:58.951201 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:58.951164    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:04:59.668637 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:04:59.668602    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:00.354427 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:00.354392    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:01.142752 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:01.142717    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:01.802502 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:01.802466    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:02.637337 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:02.637306    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:12.859182 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:12.859144    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"]
Sep 18 19:05:12.859629 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:12.859419    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5" podUID="b49feb8e-662b-4471-99cc-8ae5dc78438a" containerName="maas-api" containerID="cri-o://fe207efd0d200729cc4edd3641a716da23f081bf8a1e6c37ce6612766465c2e4" gracePeriod=30
Sep 18 19:05:12.877661 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:12.877595    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:12.917896 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:12.917866    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk"]
Sep 18 19:05:12.918147 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:12.918115    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk" podUID="f44d9f25-26a3-4cb4-885c-816a45dbbc1c" containerName="payload-processing" containerID="cri-o://1173e1115dad77cf9f1b6db29aefae74204d064c08d4c5639f20959502f3f24a" gracePeriod=30
Sep 18 19:05:12.922230 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:12.922204    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd"]
Sep 18 19:05:12.922682 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:12.922478    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd" podUID="b573a274-0f3c-4cdc-a7ce-19cbb152ce8a" containerName="payload-pre-processing" containerID="cri-o://db5c8974f94525ef51f50c91fa30e930f30f291438978c6eda6085363bb59569" gracePeriod=30
Sep 18 19:05:12.971231 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:12.971194    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:13.042832 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.042782    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:13.192470 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.192443    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"
Sep 18 19:05:13.222286 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.222257    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk"
Sep 18 19:05:13.225797 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.225770    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd"
Sep 18 19:05:13.285910 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.285884    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b49feb8e-662b-4471-99cc-8ae5dc78438a-metrics-certs\") pod \"b49feb8e-662b-4471-99cc-8ae5dc78438a\" (UID: \"b49feb8e-662b-4471-99cc-8ae5dc78438a\") "
Sep 18 19:05:13.286097 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.285925    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b49feb8e-662b-4471-99cc-8ae5dc78438a-openshift-service-ca\") pod \"b49feb8e-662b-4471-99cc-8ae5dc78438a\" (UID: \"b49feb8e-662b-4471-99cc-8ae5dc78438a\") "
Sep 18 19:05:13.286097 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.285980    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrnsz\" (UniqueName: \"kubernetes.io/projected/f44d9f25-26a3-4cb4-885c-816a45dbbc1c-kube-api-access-nrnsz\") pod \"f44d9f25-26a3-4cb4-885c-816a45dbbc1c\" (UID: \"f44d9f25-26a3-4cb4-885c-816a45dbbc1c\") "
Sep 18 19:05:13.286097 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.286006    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b573a274-0f3c-4cdc-a7ce-19cbb152ce8a-plugins-config-volume\") pod \"b573a274-0f3c-4cdc-a7ce-19cbb152ce8a\" (UID: \"b573a274-0f3c-4cdc-a7ce-19cbb152ce8a\") "
Sep 18 19:05:13.286097 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.286054    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f44d9f25-26a3-4cb4-885c-816a45dbbc1c-plugins-config-volume\") pod \"f44d9f25-26a3-4cb4-885c-816a45dbbc1c\" (UID: \"f44d9f25-26a3-4cb4-885c-816a45dbbc1c\") "
Sep 18 19:05:13.286097 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.286094    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8kt6\" (UniqueName: \"kubernetes.io/projected/b573a274-0f3c-4cdc-a7ce-19cbb152ce8a-kube-api-access-d8kt6\") pod \"b573a274-0f3c-4cdc-a7ce-19cbb152ce8a\" (UID: \"b573a274-0f3c-4cdc-a7ce-19cbb152ce8a\") "
Sep 18 19:05:13.286367 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.286166    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqbgh\" (UniqueName: \"kubernetes.io/projected/b49feb8e-662b-4471-99cc-8ae5dc78438a-kube-api-access-bqbgh\") pod \"b49feb8e-662b-4471-99cc-8ae5dc78438a\" (UID: \"b49feb8e-662b-4471-99cc-8ae5dc78438a\") "
Sep 18 19:05:13.286367 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.286193    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b49feb8e-662b-4471-99cc-8ae5dc78438a-maas-api-tls\") pod \"b49feb8e-662b-4471-99cc-8ae5dc78438a\" (UID: \"b49feb8e-662b-4471-99cc-8ae5dc78438a\") "
Sep 18 19:05:13.286474 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.286377    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b49feb8e-662b-4471-99cc-8ae5dc78438a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b49feb8e-662b-4471-99cc-8ae5dc78438a" (UID: "b49feb8e-662b-4471-99cc-8ae5dc78438a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:05:13.286567 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.286373    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b573a274-0f3c-4cdc-a7ce-19cbb152ce8a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b573a274-0f3c-4cdc-a7ce-19cbb152ce8a" (UID: "b573a274-0f3c-4cdc-a7ce-19cbb152ce8a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:05:13.286637 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.286554    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b49feb8e-662b-4471-99cc-8ae5dc78438a-openshift-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:13.286770 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.286743    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f44d9f25-26a3-4cb4-885c-816a45dbbc1c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f44d9f25-26a3-4cb4-885c-816a45dbbc1c" (UID: "f44d9f25-26a3-4cb4-885c-816a45dbbc1c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:05:13.288341 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.288311    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f44d9f25-26a3-4cb4-885c-816a45dbbc1c-kube-api-access-nrnsz" (OuterVolumeSpecName: "kube-api-access-nrnsz") pod "f44d9f25-26a3-4cb4-885c-816a45dbbc1c" (UID: "f44d9f25-26a3-4cb4-885c-816a45dbbc1c"). InnerVolumeSpecName "kube-api-access-nrnsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:05:13.288441 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.288368    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b49feb8e-662b-4471-99cc-8ae5dc78438a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b49feb8e-662b-4471-99cc-8ae5dc78438a" (UID: "b49feb8e-662b-4471-99cc-8ae5dc78438a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:05:13.288579 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.288555    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b49feb8e-662b-4471-99cc-8ae5dc78438a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b49feb8e-662b-4471-99cc-8ae5dc78438a" (UID: "b49feb8e-662b-4471-99cc-8ae5dc78438a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:05:13.288678 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.288616    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b49feb8e-662b-4471-99cc-8ae5dc78438a-kube-api-access-bqbgh" (OuterVolumeSpecName: "kube-api-access-bqbgh") pod "b49feb8e-662b-4471-99cc-8ae5dc78438a" (UID: "b49feb8e-662b-4471-99cc-8ae5dc78438a"). InnerVolumeSpecName "kube-api-access-bqbgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:05:13.288777 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.288760    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b573a274-0f3c-4cdc-a7ce-19cbb152ce8a-kube-api-access-d8kt6" (OuterVolumeSpecName: "kube-api-access-d8kt6") pod "b573a274-0f3c-4cdc-a7ce-19cbb152ce8a" (UID: "b573a274-0f3c-4cdc-a7ce-19cbb152ce8a"). InnerVolumeSpecName "kube-api-access-d8kt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:05:13.387098 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.387028    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8kt6\" (UniqueName: \"kubernetes.io/projected/b573a274-0f3c-4cdc-a7ce-19cbb152ce8a-kube-api-access-d8kt6\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:13.387098 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.387053    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqbgh\" (UniqueName: \"kubernetes.io/projected/b49feb8e-662b-4471-99cc-8ae5dc78438a-kube-api-access-bqbgh\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:13.387098 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.387067    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b49feb8e-662b-4471-99cc-8ae5dc78438a-maas-api-tls\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:13.387098 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.387080    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b49feb8e-662b-4471-99cc-8ae5dc78438a-metrics-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:13.387098 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.387091    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrnsz\" (UniqueName: \"kubernetes.io/projected/f44d9f25-26a3-4cb4-885c-816a45dbbc1c-kube-api-access-nrnsz\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:13.387352 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.387103    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b573a274-0f3c-4cdc-a7ce-19cbb152ce8a-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:13.387352 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.387116    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f44d9f25-26a3-4cb4-885c-816a45dbbc1c-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:13.835691 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.835659    2558 generic.go:356] "Generic (PLEG): container finished" podID="b573a274-0f3c-4cdc-a7ce-19cbb152ce8a" containerID="db5c8974f94525ef51f50c91fa30e930f30f291438978c6eda6085363bb59569" exitCode=0
Sep 18 19:05:13.835865 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.835703    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd"
Sep 18 19:05:13.835865 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.835745    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd" event={"ID":"b573a274-0f3c-4cdc-a7ce-19cbb152ce8a","Type":"ContainerDied","Data":"db5c8974f94525ef51f50c91fa30e930f30f291438978c6eda6085363bb59569"}
Sep 18 19:05:13.835865 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.835803    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd" event={"ID":"b573a274-0f3c-4cdc-a7ce-19cbb152ce8a","Type":"ContainerDied","Data":"393286f5660dd8febf9f545b779fe9f2362667650faa1f04f10604772459faf8"}
Sep 18 19:05:13.835865 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.835857    2558 scope.go:117] "RemoveContainer" containerID="db5c8974f94525ef51f50c91fa30e930f30f291438978c6eda6085363bb59569"
Sep 18 19:05:13.837077 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.837061    2558 generic.go:356] "Generic (PLEG): container finished" podID="b49feb8e-662b-4471-99cc-8ae5dc78438a" containerID="fe207efd0d200729cc4edd3641a716da23f081bf8a1e6c37ce6612766465c2e4" exitCode=0
Sep 18 19:05:13.837154 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.837108    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"
Sep 18 19:05:13.837206 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.837150    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5" event={"ID":"b49feb8e-662b-4471-99cc-8ae5dc78438a","Type":"ContainerDied","Data":"fe207efd0d200729cc4edd3641a716da23f081bf8a1e6c37ce6612766465c2e4"}
Sep 18 19:05:13.837267 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.837206    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5" event={"ID":"b49feb8e-662b-4471-99cc-8ae5dc78438a","Type":"ContainerDied","Data":"1c9279a43ba874dc95934e42e99ffc8f2906c5781c5810303990b1f5cb254335"}
Sep 18 19:05:13.838705 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.838683    2558 generic.go:356] "Generic (PLEG): container finished" podID="f44d9f25-26a3-4cb4-885c-816a45dbbc1c" containerID="1173e1115dad77cf9f1b6db29aefae74204d064c08d4c5639f20959502f3f24a" exitCode=0
Sep 18 19:05:13.838792 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.838780    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk"
Sep 18 19:05:13.846960 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.846942    2558 scope.go:117] "RemoveContainer" containerID="db5c8974f94525ef51f50c91fa30e930f30f291438978c6eda6085363bb59569"
Sep 18 19:05:13.847334 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:05:13.847253    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db5c8974f94525ef51f50c91fa30e930f30f291438978c6eda6085363bb59569\": container with ID starting with db5c8974f94525ef51f50c91fa30e930f30f291438978c6eda6085363bb59569 not found: ID does not exist" containerID="db5c8974f94525ef51f50c91fa30e930f30f291438978c6eda6085363bb59569"
Sep 18 19:05:13.847334 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.847278    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db5c8974f94525ef51f50c91fa30e930f30f291438978c6eda6085363bb59569"} err="failed to get container status \"db5c8974f94525ef51f50c91fa30e930f30f291438978c6eda6085363bb59569\": rpc error: code = NotFound desc = could not find container \"db5c8974f94525ef51f50c91fa30e930f30f291438978c6eda6085363bb59569\": container with ID starting with db5c8974f94525ef51f50c91fa30e930f30f291438978c6eda6085363bb59569 not found: ID does not exist"
Sep 18 19:05:13.847334 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.847295    2558 scope.go:117] "RemoveContainer" containerID="fe207efd0d200729cc4edd3641a716da23f081bf8a1e6c37ce6612766465c2e4"
Sep 18 19:05:13.850400 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.850376    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk" event={"ID":"f44d9f25-26a3-4cb4-885c-816a45dbbc1c","Type":"ContainerDied","Data":"1173e1115dad77cf9f1b6db29aefae74204d064c08d4c5639f20959502f3f24a"}
Sep 18 19:05:13.850513 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.850425    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk" event={"ID":"f44d9f25-26a3-4cb4-885c-816a45dbbc1c","Type":"ContainerDied","Data":"83c6e70efd74c708508751cb276020b4ed079ea998d0fb6dfd3b665a5236127d"}
Sep 18 19:05:13.858481 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.858464    2558 scope.go:117] "RemoveContainer" containerID="fe207efd0d200729cc4edd3641a716da23f081bf8a1e6c37ce6612766465c2e4"
Sep 18 19:05:13.858728 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:05:13.858712    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe207efd0d200729cc4edd3641a716da23f081bf8a1e6c37ce6612766465c2e4\": container with ID starting with fe207efd0d200729cc4edd3641a716da23f081bf8a1e6c37ce6612766465c2e4 not found: ID does not exist" containerID="fe207efd0d200729cc4edd3641a716da23f081bf8a1e6c37ce6612766465c2e4"
Sep 18 19:05:13.858777 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.858733    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe207efd0d200729cc4edd3641a716da23f081bf8a1e6c37ce6612766465c2e4"} err="failed to get container status \"fe207efd0d200729cc4edd3641a716da23f081bf8a1e6c37ce6612766465c2e4\": rpc error: code = NotFound desc = could not find container \"fe207efd0d200729cc4edd3641a716da23f081bf8a1e6c37ce6612766465c2e4\": container with ID starting with fe207efd0d200729cc4edd3641a716da23f081bf8a1e6c37ce6612766465c2e4 not found: ID does not exist"
Sep 18 19:05:13.858777 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.858748    2558 scope.go:117] "RemoveContainer" containerID="1173e1115dad77cf9f1b6db29aefae74204d064c08d4c5639f20959502f3f24a"
Sep 18 19:05:13.866760 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.866623    2558 scope.go:117] "RemoveContainer" containerID="1173e1115dad77cf9f1b6db29aefae74204d064c08d4c5639f20959502f3f24a"
Sep 18 19:05:13.867053 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:05:13.866894    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1173e1115dad77cf9f1b6db29aefae74204d064c08d4c5639f20959502f3f24a\": container with ID starting with 1173e1115dad77cf9f1b6db29aefae74204d064c08d4c5639f20959502f3f24a not found: ID does not exist" containerID="1173e1115dad77cf9f1b6db29aefae74204d064c08d4c5639f20959502f3f24a"
Sep 18 19:05:13.867053 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.866928    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1173e1115dad77cf9f1b6db29aefae74204d064c08d4c5639f20959502f3f24a"} err="failed to get container status \"1173e1115dad77cf9f1b6db29aefae74204d064c08d4c5639f20959502f3f24a\": rpc error: code = NotFound desc = could not find container \"1173e1115dad77cf9f1b6db29aefae74204d064c08d4c5639f20959502f3f24a\": container with ID starting with 1173e1115dad77cf9f1b6db29aefae74204d064c08d4c5639f20959502f3f24a not found: ID does not exist"
Sep 18 19:05:13.867630 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.867614    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"]
Sep 18 19:05:13.871804 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.871786    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ce3141-68c857f595-8zxc5"]
Sep 18 19:05:13.887485 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.887459    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd"]
Sep 18 19:05:13.889746 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.889722    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ce3141-b45fdbcb6-c9mwd"]
Sep 18 19:05:13.905675 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.905651    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk"]
Sep 18 19:05:13.912848 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.912811    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ce3141-6f97b9cb57-hj8qk"]
Sep 18 19:05:13.916695 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.916640    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:13.982731 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:13.982704    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:14.611707 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:14.611674    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:15.338068 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:15.338036    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:15.845679 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:15.845646    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b49feb8e-662b-4471-99cc-8ae5dc78438a" path="/var/lib/kubelet/pods/b49feb8e-662b-4471-99cc-8ae5dc78438a/volumes"
Sep 18 19:05:15.846237 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:15.846207    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b573a274-0f3c-4cdc-a7ce-19cbb152ce8a" path="/var/lib/kubelet/pods/b573a274-0f3c-4cdc-a7ce-19cbb152ce8a/volumes"
Sep 18 19:05:15.846721 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:15.846699    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f44d9f25-26a3-4cb4-885c-816a45dbbc1c" path="/var/lib/kubelet/pods/f44d9f25-26a3-4cb4-885c-816a45dbbc1c/volumes"
Sep 18 19:05:16.023146 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:16.023112    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:16.853517 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:16.853481    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:17.612365 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:17.612331    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:19.465228 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:19.465194    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:19.569132 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:19.569100    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:19.700195 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:19.700161    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:20.153759 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:20.153723    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:20.909385 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:20.909355    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:21.732494 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:21.732462    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:22.533598 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:22.533532    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:23.414445 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:23.414413    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:23.929450 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:23.929418    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:24.097369 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.097342    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8953-699079"]
Sep 18 19:05:24.097813 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.097799    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b573a274-0f3c-4cdc-a7ce-19cbb152ce8a" containerName="payload-pre-processing"
Sep 18 19:05:24.097813 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.097815    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b573a274-0f3c-4cdc-a7ce-19cbb152ce8a" containerName="payload-pre-processing"
Sep 18 19:05:24.097912 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.097834    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f44d9f25-26a3-4cb4-885c-816a45dbbc1c" containerName="payload-processing"
Sep 18 19:05:24.097912 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.097840    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f44d9f25-26a3-4cb4-885c-816a45dbbc1c" containerName="payload-processing"
Sep 18 19:05:24.097912 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.097850    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b49feb8e-662b-4471-99cc-8ae5dc78438a" containerName="maas-api"
Sep 18 19:05:24.097912 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.097856    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b49feb8e-662b-4471-99cc-8ae5dc78438a" containerName="maas-api"
Sep 18 19:05:24.098042 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.097922    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b49feb8e-662b-4471-99cc-8ae5dc78438a" containerName="maas-api"
Sep 18 19:05:24.098042 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.097929    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f44d9f25-26a3-4cb4-885c-816a45dbbc1c" containerName="payload-processing"
Sep 18 19:05:24.098042 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.097940    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b573a274-0f3c-4cdc-a7ce-19cbb152ce8a" containerName="payload-pre-processing"
Sep 18 19:05:24.102172 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.102153    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8953-699079"
Sep 18 19:05:24.105202 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.105176    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7m4wl\""
Sep 18 19:05:24.107373 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.107349    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8953-699079"]
Sep 18 19:05:24.187279 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.187200    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx6db\" (UniqueName: \"kubernetes.io/projected/2c3e22aa-a539-425f-8173-6434441f0f48-kube-api-access-cx6db\") pod \"test-curl-8953-699079\" (UID: \"2c3e22aa-a539-425f-8173-6434441f0f48\") " pod="opendatahub/test-curl-8953-699079"
Sep 18 19:05:24.288469 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.288433    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx6db\" (UniqueName: \"kubernetes.io/projected/2c3e22aa-a539-425f-8173-6434441f0f48-kube-api-access-cx6db\") pod \"test-curl-8953-699079\" (UID: \"2c3e22aa-a539-425f-8173-6434441f0f48\") " pod="opendatahub/test-curl-8953-699079"
Sep 18 19:05:24.297025 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.296995    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx6db\" (UniqueName: \"kubernetes.io/projected/2c3e22aa-a539-425f-8173-6434441f0f48-kube-api-access-cx6db\") pod \"test-curl-8953-699079\" (UID: \"2c3e22aa-a539-425f-8173-6434441f0f48\") " pod="opendatahub/test-curl-8953-699079"
Sep 18 19:05:24.413224 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.413193    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8953-699079"
Sep 18 19:05:24.683911 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.683878    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8953-699079"]
Sep 18 19:05:24.687174 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:05:24.687150    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c3e22aa_a539_425f_8173_6434441f0f48.slice/crio-fb9677425e701259e295a6ad92815ab0ed21fda2b6ecea1a74e40441a9df735c WatchSource:0}: Error finding container fb9677425e701259e295a6ad92815ab0ed21fda2b6ecea1a74e40441a9df735c: Status 404 returned error can't find the container with id fb9677425e701259e295a6ad92815ab0ed21fda2b6ecea1a74e40441a9df735c
Sep 18 19:05:24.709198 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.709171    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:24.899029 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.898990    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8953-699079" event={"ID":"2c3e22aa-a539-425f-8173-6434441f0f48","Type":"ContainerStarted","Data":"a752958d17ba4d11723eb7ec1c84e6c2b9356ab60274abbc53a7c450df827da6"}
Sep 18 19:05:24.899169 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.899038    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8953-699079" event={"ID":"2c3e22aa-a539-425f-8173-6434441f0f48","Type":"ContainerStarted","Data":"fb9677425e701259e295a6ad92815ab0ed21fda2b6ecea1a74e40441a9df735c"}
Sep 18 19:05:24.914388 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:24.914346    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8953-699079" podStartSLOduration=0.9143305 podStartE2EDuration="914.3305ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:05:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:05:24.913214435 +0000 UTC m=+1561.111698640" watchObservedRunningTime="2026-09-18 19:05:24.9143305 +0000 UTC m=+1561.112814705"
Sep 18 19:05:25.504072 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:25.504034    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8953-699079"]
Sep 18 19:05:25.512327 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:25.512296    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8953-699079"]
Sep 18 19:05:25.631637 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:25.631609    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8953-62238a"]
Sep 18 19:05:25.632080 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:25.632067    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c3e22aa-a539-425f-8173-6434441f0f48" containerName="test-curl-8953-699079"
Sep 18 19:05:25.632134 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:25.632082    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c3e22aa-a539-425f-8173-6434441f0f48" containerName="test-curl-8953-699079"
Sep 18 19:05:25.632175 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:25.632149    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c3e22aa-a539-425f-8173-6434441f0f48" containerName="test-curl-8953-699079"
Sep 18 19:05:25.635453 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:25.635424    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8953-62238a"
Sep 18 19:05:25.643414 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:25.643083    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8953-62238a"]
Sep 18 19:05:25.701060 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:25.701036    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbwlm\" (UniqueName: \"kubernetes.io/projected/e4b2278c-a55a-43cf-9c36-528713c72556-kube-api-access-jbwlm\") pod \"test-curl-8953-62238a\" (UID: \"e4b2278c-a55a-43cf-9c36-528713c72556\") " pod="opendatahub/test-curl-8953-62238a"
Sep 18 19:05:25.801709 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:25.801637    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbwlm\" (UniqueName: \"kubernetes.io/projected/e4b2278c-a55a-43cf-9c36-528713c72556-kube-api-access-jbwlm\") pod \"test-curl-8953-62238a\" (UID: \"e4b2278c-a55a-43cf-9c36-528713c72556\") " pod="opendatahub/test-curl-8953-62238a"
Sep 18 19:05:25.810134 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:25.810106    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbwlm\" (UniqueName: \"kubernetes.io/projected/e4b2278c-a55a-43cf-9c36-528713c72556-kube-api-access-jbwlm\") pod \"test-curl-8953-62238a\" (UID: \"e4b2278c-a55a-43cf-9c36-528713c72556\") " pod="opendatahub/test-curl-8953-62238a"
Sep 18 19:05:25.906701 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:05:25.906671    2558 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8953-699079" container="test-curl-8953-699079"
Sep 18 19:05:25.946923 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:25.946896    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8953-62238a"
Sep 18 19:05:26.215274 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:26.215248    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8953-62238a"]
Sep 18 19:05:26.217070 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:05:26.217038    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4b2278c_a55a_43cf_9c36_528713c72556.slice/crio-17778e69dad73c8fe736cc3db163b9f99c033a9580d082e25aaede6255ca1cc9 WatchSource:0}: Error finding container 17778e69dad73c8fe736cc3db163b9f99c033a9580d082e25aaede6255ca1cc9: Status 404 returned error can't find the container with id 17778e69dad73c8fe736cc3db163b9f99c033a9580d082e25aaede6255ca1cc9
Sep 18 19:05:26.913108 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:26.913067    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8953-62238a" event={"ID":"e4b2278c-a55a-43cf-9c36-528713c72556","Type":"ContainerStarted","Data":"374803934e6ad92f59223ccb81b75711be41be04b26a0863cdd6a20abe834536"}
Sep 18 19:05:26.913568 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:26.913122    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8953-62238a" event={"ID":"e4b2278c-a55a-43cf-9c36-528713c72556","Type":"ContainerStarted","Data":"17778e69dad73c8fe736cc3db163b9f99c033a9580d082e25aaede6255ca1cc9"}
Sep 18 19:05:26.913568 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:26.913160    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8953-699079" podUID="2c3e22aa-a539-425f-8173-6434441f0f48" containerName="test-curl-8953-699079" containerID="cri-o://a752958d17ba4d11723eb7ec1c84e6c2b9356ab60274abbc53a7c450df827da6" gracePeriod=2
Sep 18 19:05:26.927794 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:26.927748    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8953-62238a" podStartSLOduration=1.9277331420000001 podStartE2EDuration="1.927733142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:05:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:05:26.926420542 +0000 UTC m=+1563.124904744" watchObservedRunningTime="2026-09-18 19:05:26.927733142 +0000 UTC m=+1563.126217347"
Sep 18 19:05:27.490629 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:27.490596    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8953-62238a"]
Sep 18 19:05:27.494804 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:27.494780    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8953-62238a"]
Sep 18 19:05:27.918104 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:05:27.918077    2558 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8953-62238a" container="test-curl-8953-62238a"
Sep 18 19:05:28.923660 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:28.923627    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8953-62238a" podUID="e4b2278c-a55a-43cf-9c36-528713c72556" containerName="test-curl-8953-62238a" containerID="cri-o://374803934e6ad92f59223ccb81b75711be41be04b26a0863cdd6a20abe834536" gracePeriod=2
Sep 18 19:05:29.095230 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:29.095209    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8953-699079"
Sep 18 19:05:29.235200 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:29.235115    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cx6db\" (UniqueName: \"kubernetes.io/projected/2c3e22aa-a539-425f-8173-6434441f0f48-kube-api-access-cx6db\") pod \"2c3e22aa-a539-425f-8173-6434441f0f48\" (UID: \"2c3e22aa-a539-425f-8173-6434441f0f48\") "
Sep 18 19:05:29.237129 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:29.237096    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c3e22aa-a539-425f-8173-6434441f0f48-kube-api-access-cx6db" (OuterVolumeSpecName: "kube-api-access-cx6db") pod "2c3e22aa-a539-425f-8173-6434441f0f48" (UID: "2c3e22aa-a539-425f-8173-6434441f0f48"). InnerVolumeSpecName "kube-api-access-cx6db". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:05:29.336909 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:29.336880    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx6db\" (UniqueName: \"kubernetes.io/projected/2c3e22aa-a539-425f-8173-6434441f0f48-kube-api-access-cx6db\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:29.467483 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:29.467456    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:29.525704 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:29.525674    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:29.851406 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:29.851329    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c3e22aa-a539-425f-8173-6434441f0f48" path="/var/lib/kubelet/pods/2c3e22aa-a539-425f-8173-6434441f0f48/volumes"
Sep 18 19:05:29.930392 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:29.930363    2558 generic.go:356] "Generic (PLEG): container finished" podID="2c3e22aa-a539-425f-8173-6434441f0f48" containerID="a752958d17ba4d11723eb7ec1c84e6c2b9356ab60274abbc53a7c450df827da6" exitCode=137
Sep 18 19:05:29.930756 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:29.930421    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8953-699079"
Sep 18 19:05:29.930756 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:29.930458    2558 scope.go:117] "RemoveContainer" containerID="a752958d17ba4d11723eb7ec1c84e6c2b9356ab60274abbc53a7c450df827da6"
Sep 18 19:05:29.939112 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:29.939091    2558 scope.go:117] "RemoveContainer" containerID="a752958d17ba4d11723eb7ec1c84e6c2b9356ab60274abbc53a7c450df827da6"
Sep 18 19:05:29.939392 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:05:29.939373    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a752958d17ba4d11723eb7ec1c84e6c2b9356ab60274abbc53a7c450df827da6\": container with ID starting with a752958d17ba4d11723eb7ec1c84e6c2b9356ab60274abbc53a7c450df827da6 not found: ID does not exist" containerID="a752958d17ba4d11723eb7ec1c84e6c2b9356ab60274abbc53a7c450df827da6"
Sep 18 19:05:29.939450 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:29.939400    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a752958d17ba4d11723eb7ec1c84e6c2b9356ab60274abbc53a7c450df827da6"} err="failed to get container status \"a752958d17ba4d11723eb7ec1c84e6c2b9356ab60274abbc53a7c450df827da6\": rpc error: code = NotFound desc = could not find container \"a752958d17ba4d11723eb7ec1c84e6c2b9356ab60274abbc53a7c450df827da6\": container with ID starting with a752958d17ba4d11723eb7ec1c84e6c2b9356ab60274abbc53a7c450df827da6 not found: ID does not exist"
Sep 18 19:05:30.377439 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:30.377402    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:31.099312 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:31.099284    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8953-62238a"
Sep 18 19:05:31.155516 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:31.155490    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbwlm\" (UniqueName: \"kubernetes.io/projected/e4b2278c-a55a-43cf-9c36-528713c72556-kube-api-access-jbwlm\") pod \"e4b2278c-a55a-43cf-9c36-528713c72556\" (UID: \"e4b2278c-a55a-43cf-9c36-528713c72556\") "
Sep 18 19:05:31.157426 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:31.157387    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4b2278c-a55a-43cf-9c36-528713c72556-kube-api-access-jbwlm" (OuterVolumeSpecName: "kube-api-access-jbwlm") pod "e4b2278c-a55a-43cf-9c36-528713c72556" (UID: "e4b2278c-a55a-43cf-9c36-528713c72556"). InnerVolumeSpecName "kube-api-access-jbwlm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:05:31.256269 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:31.256243    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbwlm\" (UniqueName: \"kubernetes.io/projected/e4b2278c-a55a-43cf-9c36-528713c72556-kube-api-access-jbwlm\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:31.846125 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:31.846091    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4b2278c-a55a-43cf-9c36-528713c72556" path="/var/lib/kubelet/pods/e4b2278c-a55a-43cf-9c36-528713c72556/volumes"
Sep 18 19:05:31.943620 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:31.943582    2558 generic.go:356] "Generic (PLEG): container finished" podID="e4b2278c-a55a-43cf-9c36-528713c72556" containerID="374803934e6ad92f59223ccb81b75711be41be04b26a0863cdd6a20abe834536" exitCode=137
Sep 18 19:05:31.943620 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:31.943623    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8953-62238a"
Sep 18 19:05:31.943854 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:31.943668    2558 scope.go:117] "RemoveContainer" containerID="374803934e6ad92f59223ccb81b75711be41be04b26a0863cdd6a20abe834536"
Sep 18 19:05:31.951923 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:31.951906    2558 scope.go:117] "RemoveContainer" containerID="374803934e6ad92f59223ccb81b75711be41be04b26a0863cdd6a20abe834536"
Sep 18 19:05:31.952167 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:05:31.952138    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"374803934e6ad92f59223ccb81b75711be41be04b26a0863cdd6a20abe834536\": container with ID starting with 374803934e6ad92f59223ccb81b75711be41be04b26a0863cdd6a20abe834536 not found: ID does not exist" containerID="374803934e6ad92f59223ccb81b75711be41be04b26a0863cdd6a20abe834536"
Sep 18 19:05:31.952271 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:31.952172    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"374803934e6ad92f59223ccb81b75711be41be04b26a0863cdd6a20abe834536"} err="failed to get container status \"374803934e6ad92f59223ccb81b75711be41be04b26a0863cdd6a20abe834536\": rpc error: code = NotFound desc = could not find container \"374803934e6ad92f59223ccb81b75711be41be04b26a0863cdd6a20abe834536\": container with ID starting with 374803934e6ad92f59223ccb81b75711be41be04b26a0863cdd6a20abe834536 not found: ID does not exist"
Sep 18 19:05:33.710023 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:33.709979    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:35.670592 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:35.670547    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"]
Sep 18 19:05:35.671108 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:35.670801    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5" podUID="4c0ebd9e-54bb-46a2-b81c-8f331512e0d6" containerName="maas-api" containerID="cri-o://2d5797cffa583180cd903c3829c83452129bbbbe36a622ffd3800a517a025a7d" gracePeriod=30
Sep 18 19:05:35.705118 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:35.705079    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:35.776788 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:35.776754    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:35.852190 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:35.852154    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:35.923197 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:35.923139    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:05:35.940363 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:35.940314    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:35.966371 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:35.966336    2558 generic.go:356] "Generic (PLEG): container finished" podID="4c0ebd9e-54bb-46a2-b81c-8f331512e0d6" containerID="2d5797cffa583180cd903c3829c83452129bbbbe36a622ffd3800a517a025a7d" exitCode=0
Sep 18 19:05:35.966540 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:35.966390    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"
Sep 18 19:05:35.966540 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:35.966405    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5" event={"ID":"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6","Type":"ContainerDied","Data":"2d5797cffa583180cd903c3829c83452129bbbbe36a622ffd3800a517a025a7d"}
Sep 18 19:05:35.966540 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:35.966461    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5" event={"ID":"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6","Type":"ContainerDied","Data":"0e91aeb4a6ff2aa0311f2b252890512e4e7e3ff85e3407191b6162bf187b4418"}
Sep 18 19:05:35.966540 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:35.966494    2558 scope.go:117] "RemoveContainer" containerID="2d5797cffa583180cd903c3829c83452129bbbbe36a622ffd3800a517a025a7d"
Sep 18 19:05:35.975593 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:35.975576    2558 scope.go:117] "RemoveContainer" containerID="2d5797cffa583180cd903c3829c83452129bbbbe36a622ffd3800a517a025a7d"
Sep 18 19:05:35.975899 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:05:35.975878    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d5797cffa583180cd903c3829c83452129bbbbe36a622ffd3800a517a025a7d\": container with ID starting with 2d5797cffa583180cd903c3829c83452129bbbbe36a622ffd3800a517a025a7d not found: ID does not exist" containerID="2d5797cffa583180cd903c3829c83452129bbbbe36a622ffd3800a517a025a7d"
Sep 18 19:05:35.975973 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:35.975911    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d5797cffa583180cd903c3829c83452129bbbbe36a622ffd3800a517a025a7d"} err="failed to get container status \"2d5797cffa583180cd903c3829c83452129bbbbe36a622ffd3800a517a025a7d\": rpc error: code = NotFound desc = could not find container \"2d5797cffa583180cd903c3829c83452129bbbbe36a622ffd3800a517a025a7d\": container with ID starting with 2d5797cffa583180cd903c3829c83452129bbbbe36a622ffd3800a517a025a7d not found: ID does not exist"
Sep 18 19:05:36.003420 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:36.003390    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:36.005505 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:36.005484    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-openshift-service-ca\") pod \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\" (UID: \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\") "
Sep 18 19:05:36.005615 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:36.005596    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-metrics-certs\") pod \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\" (UID: \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\") "
Sep 18 19:05:36.005661 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:36.005629    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4k9m\" (UniqueName: \"kubernetes.io/projected/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-kube-api-access-k4k9m\") pod \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\" (UID: \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\") "
Sep 18 19:05:36.005706 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:36.005676    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-maas-api-tls\") pod \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\" (UID: \"4c0ebd9e-54bb-46a2-b81c-8f331512e0d6\") "
Sep 18 19:05:36.005861 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:36.005811    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4c0ebd9e-54bb-46a2-b81c-8f331512e0d6" (UID: "4c0ebd9e-54bb-46a2-b81c-8f331512e0d6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:05:36.006001 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:36.005981    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-openshift-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:36.007613 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:36.007579    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4c0ebd9e-54bb-46a2-b81c-8f331512e0d6" (UID: "4c0ebd9e-54bb-46a2-b81c-8f331512e0d6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:05:36.007842 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:36.007794    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4c0ebd9e-54bb-46a2-b81c-8f331512e0d6" (UID: "4c0ebd9e-54bb-46a2-b81c-8f331512e0d6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:05:36.007842 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:36.007806    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-kube-api-access-k4k9m" (OuterVolumeSpecName: "kube-api-access-k4k9m") pod "4c0ebd9e-54bb-46a2-b81c-8f331512e0d6" (UID: "4c0ebd9e-54bb-46a2-b81c-8f331512e0d6"). InnerVolumeSpecName "kube-api-access-k4k9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:05:36.107145 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:36.107115    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-maas-api-tls\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:36.107145 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:36.107140    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-metrics-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:36.107340 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:36.107154    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4k9m\" (UniqueName: \"kubernetes.io/projected/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6-kube-api-access-k4k9m\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:36.220024 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:36.219929    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:36.289177 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:36.289147    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"]
Sep 18 19:05:36.292058 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:36.292026    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:36.293405 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:36.293385    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-02e2c9-855f98db96-k7vn5"]
Sep 18 19:05:37.022097 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:37.022069    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:37.733163 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:37.733126    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:37.845689 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:37.845653    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c0ebd9e-54bb-46a2-b81c-8f331512e0d6" path="/var/lib/kubelet/pods/4c0ebd9e-54bb-46a2-b81c-8f331512e0d6/volumes"
Sep 18 19:05:38.644871 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:38.644836    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:39.380931 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:39.380896    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:42.211973 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:42.211933    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:42.285995 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:42.285960    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:42.493805 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:42.493774    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:43.117941 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:43.117909    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:43.926297 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:43.926267    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:44.598835 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:44.598787    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:45.251801 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:45.251761    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:46.038851 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:46.038801    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:46.828217 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:46.828178    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:48.953175 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:48.953145    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:52.723087 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:52.723052    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:53.370089 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.370059    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"]
Sep 18 19:05:53.370410 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.370355    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s" podUID="d3d18575-7648-42f0-a048-ce5fe38237e9" containerName="maas-api" containerID="cri-o://c2c6655a0070039435fb6d89c7f41b732f58ccf945b0780c7200613a91e93b50" gracePeriod=30
Sep 18 19:05:53.400569 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.400537    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:53.430056 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.430014    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg"]
Sep 18 19:05:53.430293 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.430264    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg" podUID="df43d446-cb9a-4201-96e9-f40f93b98955" containerName="payload-processing" containerID="cri-o://b940fedc8adefae222cc9dced48a015b0557e44b7b5d9f49f6e491c9eeab93c9" gracePeriod=30
Sep 18 19:05:53.499388 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.499355    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:53.601401 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.601370    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:53.687795 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.687772    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 19:05:53.701334 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.701302    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:53.715113 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.715087    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg"
Sep 18 19:05:53.780300 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.780264    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3d18575-7648-42f0-a048-ce5fe38237e9-maas-api-tls\") pod \"d3d18575-7648-42f0-a048-ce5fe38237e9\" (UID: \"d3d18575-7648-42f0-a048-ce5fe38237e9\") "
Sep 18 19:05:53.780777 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.780309    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3d18575-7648-42f0-a048-ce5fe38237e9-metrics-certs\") pod \"d3d18575-7648-42f0-a048-ce5fe38237e9\" (UID: \"d3d18575-7648-42f0-a048-ce5fe38237e9\") "
Sep 18 19:05:53.780777 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.780361    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-669fc\" (UniqueName: \"kubernetes.io/projected/df43d446-cb9a-4201-96e9-f40f93b98955-kube-api-access-669fc\") pod \"df43d446-cb9a-4201-96e9-f40f93b98955\" (UID: \"df43d446-cb9a-4201-96e9-f40f93b98955\") "
Sep 18 19:05:53.780777 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.780413    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgh5t\" (UniqueName: \"kubernetes.io/projected/d3d18575-7648-42f0-a048-ce5fe38237e9-kube-api-access-wgh5t\") pod \"d3d18575-7648-42f0-a048-ce5fe38237e9\" (UID: \"d3d18575-7648-42f0-a048-ce5fe38237e9\") "
Sep 18 19:05:53.780777 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.780439    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3d18575-7648-42f0-a048-ce5fe38237e9-openshift-service-ca\") pod \"d3d18575-7648-42f0-a048-ce5fe38237e9\" (UID: \"d3d18575-7648-42f0-a048-ce5fe38237e9\") "
Sep 18 19:05:53.780777 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.780507    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df43d446-cb9a-4201-96e9-f40f93b98955-plugins-config-volume\") pod \"df43d446-cb9a-4201-96e9-f40f93b98955\" (UID: \"df43d446-cb9a-4201-96e9-f40f93b98955\") "
Sep 18 19:05:53.781105 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.780896    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3d18575-7648-42f0-a048-ce5fe38237e9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d3d18575-7648-42f0-a048-ce5fe38237e9" (UID: "d3d18575-7648-42f0-a048-ce5fe38237e9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:05:53.781105 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.780915    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df43d446-cb9a-4201-96e9-f40f93b98955-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "df43d446-cb9a-4201-96e9-f40f93b98955" (UID: "df43d446-cb9a-4201-96e9-f40f93b98955"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:05:53.782668 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.782640    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df43d446-cb9a-4201-96e9-f40f93b98955-kube-api-access-669fc" (OuterVolumeSpecName: "kube-api-access-669fc") pod "df43d446-cb9a-4201-96e9-f40f93b98955" (UID: "df43d446-cb9a-4201-96e9-f40f93b98955"). InnerVolumeSpecName "kube-api-access-669fc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:05:53.782668 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.782659    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3d18575-7648-42f0-a048-ce5fe38237e9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d3d18575-7648-42f0-a048-ce5fe38237e9" (UID: "d3d18575-7648-42f0-a048-ce5fe38237e9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:05:53.782808 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.782640    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3d18575-7648-42f0-a048-ce5fe38237e9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d3d18575-7648-42f0-a048-ce5fe38237e9" (UID: "d3d18575-7648-42f0-a048-ce5fe38237e9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:05:53.782808 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.782695    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3d18575-7648-42f0-a048-ce5fe38237e9-kube-api-access-wgh5t" (OuterVolumeSpecName: "kube-api-access-wgh5t") pod "d3d18575-7648-42f0-a048-ce5fe38237e9" (UID: "d3d18575-7648-42f0-a048-ce5fe38237e9"). InnerVolumeSpecName "kube-api-access-wgh5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:05:53.882219 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.882179    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-669fc\" (UniqueName: \"kubernetes.io/projected/df43d446-cb9a-4201-96e9-f40f93b98955-kube-api-access-669fc\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:53.882219 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.882212    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgh5t\" (UniqueName: \"kubernetes.io/projected/d3d18575-7648-42f0-a048-ce5fe38237e9-kube-api-access-wgh5t\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:53.882219 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.882225    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3d18575-7648-42f0-a048-ce5fe38237e9-openshift-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:53.882468 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.882239    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df43d446-cb9a-4201-96e9-f40f93b98955-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:53.882468 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.882252    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3d18575-7648-42f0-a048-ce5fe38237e9-maas-api-tls\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:53.882468 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.882264    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3d18575-7648-42f0-a048-ce5fe38237e9-metrics-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:05:53.974512 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:53.974431    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:54.061936 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.061899    2558 generic.go:356] "Generic (PLEG): container finished" podID="df43d446-cb9a-4201-96e9-f40f93b98955" containerID="b940fedc8adefae222cc9dced48a015b0557e44b7b5d9f49f6e491c9eeab93c9" exitCode=0
Sep 18 19:05:54.062108 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.061965    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg"
Sep 18 19:05:54.062108 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.061988    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg" event={"ID":"df43d446-cb9a-4201-96e9-f40f93b98955","Type":"ContainerDied","Data":"b940fedc8adefae222cc9dced48a015b0557e44b7b5d9f49f6e491c9eeab93c9"}
Sep 18 19:05:54.062108 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.062056    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg" event={"ID":"df43d446-cb9a-4201-96e9-f40f93b98955","Type":"ContainerDied","Data":"fb05d7f51b26fad05835cf5a73399fa2c89680bba92ab81bfd721b6c0373f55f"}
Sep 18 19:05:54.062108 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.062088    2558 scope.go:117] "RemoveContainer" containerID="b940fedc8adefae222cc9dced48a015b0557e44b7b5d9f49f6e491c9eeab93c9"
Sep 18 19:05:54.063529 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.063506    2558 generic.go:356] "Generic (PLEG): container finished" podID="d3d18575-7648-42f0-a048-ce5fe38237e9" containerID="c2c6655a0070039435fb6d89c7f41b732f58ccf945b0780c7200613a91e93b50" exitCode=0
Sep 18 19:05:54.063641 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.063536    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s" event={"ID":"d3d18575-7648-42f0-a048-ce5fe38237e9","Type":"ContainerDied","Data":"c2c6655a0070039435fb6d89c7f41b732f58ccf945b0780c7200613a91e93b50"}
Sep 18 19:05:54.063641 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.063574    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"
Sep 18 19:05:54.063641 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.063587    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s" event={"ID":"d3d18575-7648-42f0-a048-ce5fe38237e9","Type":"ContainerDied","Data":"9b626a269bef65560266e8454b90811d9423ed23ddfa6cff1788e4c1bba82a0c"}
Sep 18 19:05:54.071507 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.071491    2558 scope.go:117] "RemoveContainer" containerID="b940fedc8adefae222cc9dced48a015b0557e44b7b5d9f49f6e491c9eeab93c9"
Sep 18 19:05:54.071789 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:05:54.071763    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b940fedc8adefae222cc9dced48a015b0557e44b7b5d9f49f6e491c9eeab93c9\": container with ID starting with b940fedc8adefae222cc9dced48a015b0557e44b7b5d9f49f6e491c9eeab93c9 not found: ID does not exist" containerID="b940fedc8adefae222cc9dced48a015b0557e44b7b5d9f49f6e491c9eeab93c9"
Sep 18 19:05:54.071914 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.071799    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b940fedc8adefae222cc9dced48a015b0557e44b7b5d9f49f6e491c9eeab93c9"} err="failed to get container status \"b940fedc8adefae222cc9dced48a015b0557e44b7b5d9f49f6e491c9eeab93c9\": rpc error: code = NotFound desc = could not find container \"b940fedc8adefae222cc9dced48a015b0557e44b7b5d9f49f6e491c9eeab93c9\": container with ID starting with b940fedc8adefae222cc9dced48a015b0557e44b7b5d9f49f6e491c9eeab93c9 not found: ID does not exist"
Sep 18 19:05:54.071914 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.071834    2558 scope.go:117] "RemoveContainer" containerID="c2c6655a0070039435fb6d89c7f41b732f58ccf945b0780c7200613a91e93b50"
Sep 18 19:05:54.081152 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.081050    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg"]
Sep 18 19:05:54.081224 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.081171    2558 scope.go:117] "RemoveContainer" containerID="c2c6655a0070039435fb6d89c7f41b732f58ccf945b0780c7200613a91e93b50"
Sep 18 19:05:54.081464 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:05:54.081445    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2c6655a0070039435fb6d89c7f41b732f58ccf945b0780c7200613a91e93b50\": container with ID starting with c2c6655a0070039435fb6d89c7f41b732f58ccf945b0780c7200613a91e93b50 not found: ID does not exist" containerID="c2c6655a0070039435fb6d89c7f41b732f58ccf945b0780c7200613a91e93b50"
Sep 18 19:05:54.081509 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.081470    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2c6655a0070039435fb6d89c7f41b732f58ccf945b0780c7200613a91e93b50"} err="failed to get container status \"c2c6655a0070039435fb6d89c7f41b732f58ccf945b0780c7200613a91e93b50\": rpc error: code = NotFound desc = could not find container \"c2c6655a0070039435fb6d89c7f41b732f58ccf945b0780c7200613a91e93b50\": container with ID starting with c2c6655a0070039435fb6d89c7f41b732f58ccf945b0780c7200613a91e93b50 not found: ID does not exist"
Sep 18 19:05:54.085641 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.085622    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-880bee-7c74fcfc4c-mzsgg"]
Sep 18 19:05:54.096323 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.096303    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"]
Sep 18 19:05:54.103040 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.103019    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-880bee-7b8f84f898-qqh5s"]
Sep 18 19:05:54.122280 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.122252    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:54.716430 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:54.716324    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:55.449554 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:55.449509    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:55.844809 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:55.844774    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3d18575-7648-42f0-a048-ce5fe38237e9" path="/var/lib/kubelet/pods/d3d18575-7648-42f0-a048-ce5fe38237e9/volumes"
Sep 18 19:05:55.845277 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:55.845256    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df43d446-cb9a-4201-96e9-f40f93b98955" path="/var/lib/kubelet/pods/df43d446-cb9a-4201-96e9-f40f93b98955/volumes"
Sep 18 19:05:56.289563 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:56.289524    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:59.700095 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:59.700058    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:59.800968 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:59.800938    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:05:59.975198 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:05:59.975113    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:00.545170 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:00.545133    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:01.233847 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:01.233803    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:01.972362 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:01.972334    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:02.771600 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:02.771566    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:03.349385 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:03.349347    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:10.769662 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:10.769582    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"]
Sep 18 19:06:10.770115 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:10.769795    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh" podUID="0e1d1183-db92-40da-89f1-6b1d8302411f" containerName="maas-api" containerID="cri-o://ddaf149c05ededed0b825d5785f5eff59189371199e51b69dd4485870fcac19e" gracePeriod=30
Sep 18 19:06:10.799640 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:10.799604    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:10.800721 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:10.800695    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-351045-29829300-x9gqz"]
Sep 18 19:06:10.802574 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:10.802550    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-351045-29829300-x9gqz"]
Sep 18 19:06:10.833152 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:10.833113    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj"]
Sep 18 19:06:10.833536 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:10.833485    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj" podUID="ac14ae1c-9d31-465c-8de0-a6ab51b5e00e" containerName="payload-pre-processing" containerID="cri-o://be2ae9be566f9957c410ae46e791685a2e67cd6f8586c893e6037b8255b14f79" gracePeriod=30
Sep 18 19:06:10.897809 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:10.897777    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:10.998654 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:10.998334    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:11.093528 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.093495    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"
Sep 18 19:06:11.107131 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.107099    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:11.114455 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.114433    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj"
Sep 18 19:06:11.159838 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.159788    2558 generic.go:356] "Generic (PLEG): container finished" podID="ac14ae1c-9d31-465c-8de0-a6ab51b5e00e" containerID="be2ae9be566f9957c410ae46e791685a2e67cd6f8586c893e6037b8255b14f79" exitCode=0
Sep 18 19:06:11.160026 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.159869    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj"
Sep 18 19:06:11.160026 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.159869    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj" event={"ID":"ac14ae1c-9d31-465c-8de0-a6ab51b5e00e","Type":"ContainerDied","Data":"be2ae9be566f9957c410ae46e791685a2e67cd6f8586c893e6037b8255b14f79"}
Sep 18 19:06:11.160026 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.159978    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj" event={"ID":"ac14ae1c-9d31-465c-8de0-a6ab51b5e00e","Type":"ContainerDied","Data":"c13f2d38a1499bb837d86494c3c7fa606aad6d607c5d996ed21d1a3c85eb075b"}
Sep 18 19:06:11.160026 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.160019    2558 scope.go:117] "RemoveContainer" containerID="be2ae9be566f9957c410ae46e791685a2e67cd6f8586c893e6037b8255b14f79"
Sep 18 19:06:11.161556 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.161532    2558 generic.go:356] "Generic (PLEG): container finished" podID="0e1d1183-db92-40da-89f1-6b1d8302411f" containerID="ddaf149c05ededed0b825d5785f5eff59189371199e51b69dd4485870fcac19e" exitCode=0
Sep 18 19:06:11.161692 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.161582    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"
Sep 18 19:06:11.161692 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.161590    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh" event={"ID":"0e1d1183-db92-40da-89f1-6b1d8302411f","Type":"ContainerDied","Data":"ddaf149c05ededed0b825d5785f5eff59189371199e51b69dd4485870fcac19e"}
Sep 18 19:06:11.161692 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.161630    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh" event={"ID":"0e1d1183-db92-40da-89f1-6b1d8302411f","Type":"ContainerDied","Data":"0f33bd021eecf38f31971d637cb1dff7767835163a4c08b3061cc9ccd631c874"}
Sep 18 19:06:11.171804 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.171782    2558 scope.go:117] "RemoveContainer" containerID="be2ae9be566f9957c410ae46e791685a2e67cd6f8586c893e6037b8255b14f79"
Sep 18 19:06:11.172202 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:06:11.172170    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be2ae9be566f9957c410ae46e791685a2e67cd6f8586c893e6037b8255b14f79\": container with ID starting with be2ae9be566f9957c410ae46e791685a2e67cd6f8586c893e6037b8255b14f79 not found: ID does not exist" containerID="be2ae9be566f9957c410ae46e791685a2e67cd6f8586c893e6037b8255b14f79"
Sep 18 19:06:11.172273 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.172210    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be2ae9be566f9957c410ae46e791685a2e67cd6f8586c893e6037b8255b14f79"} err="failed to get container status \"be2ae9be566f9957c410ae46e791685a2e67cd6f8586c893e6037b8255b14f79\": rpc error: code = NotFound desc = could not find container \"be2ae9be566f9957c410ae46e791685a2e67cd6f8586c893e6037b8255b14f79\": container with ID starting with be2ae9be566f9957c410ae46e791685a2e67cd6f8586c893e6037b8255b14f79 not found: ID does not exist"
Sep 18 19:06:11.172273 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.172229    2558 scope.go:117] "RemoveContainer" containerID="ddaf149c05ededed0b825d5785f5eff59189371199e51b69dd4485870fcac19e"
Sep 18 19:06:11.180970 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.180951    2558 scope.go:117] "RemoveContainer" containerID="ddaf149c05ededed0b825d5785f5eff59189371199e51b69dd4485870fcac19e"
Sep 18 19:06:11.181309 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:06:11.181289    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddaf149c05ededed0b825d5785f5eff59189371199e51b69dd4485870fcac19e\": container with ID starting with ddaf149c05ededed0b825d5785f5eff59189371199e51b69dd4485870fcac19e not found: ID does not exist" containerID="ddaf149c05ededed0b825d5785f5eff59189371199e51b69dd4485870fcac19e"
Sep 18 19:06:11.181361 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.181318    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddaf149c05ededed0b825d5785f5eff59189371199e51b69dd4485870fcac19e"} err="failed to get container status \"ddaf149c05ededed0b825d5785f5eff59189371199e51b69dd4485870fcac19e\": rpc error: code = NotFound desc = could not find container \"ddaf149c05ededed0b825d5785f5eff59189371199e51b69dd4485870fcac19e\": container with ID starting with ddaf149c05ededed0b825d5785f5eff59189371199e51b69dd4485870fcac19e not found: ID does not exist"
Sep 18 19:06:11.253579 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.253543    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e1d1183-db92-40da-89f1-6b1d8302411f-maas-api-tls\") pod \"0e1d1183-db92-40da-89f1-6b1d8302411f\" (UID: \"0e1d1183-db92-40da-89f1-6b1d8302411f\") "
Sep 18 19:06:11.253579 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.253582    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac14ae1c-9d31-465c-8de0-a6ab51b5e00e-plugins-config-volume\") pod \"ac14ae1c-9d31-465c-8de0-a6ab51b5e00e\" (UID: \"ac14ae1c-9d31-465c-8de0-a6ab51b5e00e\") "
Sep 18 19:06:11.254370 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.253621    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jj2j5\" (UniqueName: \"kubernetes.io/projected/ac14ae1c-9d31-465c-8de0-a6ab51b5e00e-kube-api-access-jj2j5\") pod \"ac14ae1c-9d31-465c-8de0-a6ab51b5e00e\" (UID: \"ac14ae1c-9d31-465c-8de0-a6ab51b5e00e\") "
Sep 18 19:06:11.254370 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.253652    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rwrg\" (UniqueName: \"kubernetes.io/projected/0e1d1183-db92-40da-89f1-6b1d8302411f-kube-api-access-7rwrg\") pod \"0e1d1183-db92-40da-89f1-6b1d8302411f\" (UID: \"0e1d1183-db92-40da-89f1-6b1d8302411f\") "
Sep 18 19:06:11.254370 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.253679    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1d1183-db92-40da-89f1-6b1d8302411f-openshift-service-ca\") pod \"0e1d1183-db92-40da-89f1-6b1d8302411f\" (UID: \"0e1d1183-db92-40da-89f1-6b1d8302411f\") "
Sep 18 19:06:11.254370 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.253705    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e1d1183-db92-40da-89f1-6b1d8302411f-metrics-certs\") pod \"0e1d1183-db92-40da-89f1-6b1d8302411f\" (UID: \"0e1d1183-db92-40da-89f1-6b1d8302411f\") "
Sep 18 19:06:11.254370 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.254150    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e1d1183-db92-40da-89f1-6b1d8302411f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0e1d1183-db92-40da-89f1-6b1d8302411f" (UID: "0e1d1183-db92-40da-89f1-6b1d8302411f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:06:11.254370 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.254153    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac14ae1c-9d31-465c-8de0-a6ab51b5e00e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ac14ae1c-9d31-465c-8de0-a6ab51b5e00e" (UID: "ac14ae1c-9d31-465c-8de0-a6ab51b5e00e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:06:11.256095 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.256064    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e1d1183-db92-40da-89f1-6b1d8302411f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0e1d1183-db92-40da-89f1-6b1d8302411f" (UID: "0e1d1183-db92-40da-89f1-6b1d8302411f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:06:11.256371 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.256348    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e1d1183-db92-40da-89f1-6b1d8302411f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0e1d1183-db92-40da-89f1-6b1d8302411f" (UID: "0e1d1183-db92-40da-89f1-6b1d8302411f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:06:11.256458 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.256390    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e1d1183-db92-40da-89f1-6b1d8302411f-kube-api-access-7rwrg" (OuterVolumeSpecName: "kube-api-access-7rwrg") pod "0e1d1183-db92-40da-89f1-6b1d8302411f" (UID: "0e1d1183-db92-40da-89f1-6b1d8302411f"). InnerVolumeSpecName "kube-api-access-7rwrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:06:11.256458 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.256438    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac14ae1c-9d31-465c-8de0-a6ab51b5e00e-kube-api-access-jj2j5" (OuterVolumeSpecName: "kube-api-access-jj2j5") pod "ac14ae1c-9d31-465c-8de0-a6ab51b5e00e" (UID: "ac14ae1c-9d31-465c-8de0-a6ab51b5e00e"). InnerVolumeSpecName "kube-api-access-jj2j5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:06:11.355537 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.355432    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e1d1183-db92-40da-89f1-6b1d8302411f-maas-api-tls\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:06:11.355537 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.355463    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac14ae1c-9d31-465c-8de0-a6ab51b5e00e-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:06:11.355537 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.355472    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jj2j5\" (UniqueName: \"kubernetes.io/projected/ac14ae1c-9d31-465c-8de0-a6ab51b5e00e-kube-api-access-jj2j5\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:06:11.355537 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.355481    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rwrg\" (UniqueName: \"kubernetes.io/projected/0e1d1183-db92-40da-89f1-6b1d8302411f-kube-api-access-7rwrg\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:06:11.355537 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.355492    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1d1183-db92-40da-89f1-6b1d8302411f-openshift-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:06:11.355537 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.355500    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e1d1183-db92-40da-89f1-6b1d8302411f-metrics-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:06:11.367278 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.367243    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:11.467560 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.467524    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:11.482563 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.482539    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj"]
Sep 18 19:06:11.486710 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.486688    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-351045-5d5bddc8c8-8lxlj"]
Sep 18 19:06:11.497638 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.497618    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"]
Sep 18 19:06:11.501944 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.501914    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-351045-d98b9ff59-kd6kh"]
Sep 18 19:06:11.844901 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.844869    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ab9b3e9-64b0-4955-a7a4-d84cf9ca6518" path="/var/lib/kubelet/pods/0ab9b3e9-64b0-4955-a7a4-d84cf9ca6518/volumes"
Sep 18 19:06:11.845256 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.845243    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e1d1183-db92-40da-89f1-6b1d8302411f" path="/var/lib/kubelet/pods/0e1d1183-db92-40da-89f1-6b1d8302411f/volumes"
Sep 18 19:06:11.845576 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:11.845564    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac14ae1c-9d31-465c-8de0-a6ab51b5e00e" path="/var/lib/kubelet/pods/ac14ae1c-9d31-465c-8de0-a6ab51b5e00e/volumes"
Sep 18 19:06:12.353598 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:12.353557    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:13.113784 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:13.113751    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:17.245964 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:17.245929    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:17.350126 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:17.350066    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:17.490856 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:17.490795    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:26.044656 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:26.044624    2558 scope.go:117] "RemoveContainer" containerID="2cf0d03127b1ca2ff1180f6163db7b0abb07dbd7dd764d71e2528146ffacca8f"
Sep 18 19:06:27.973633 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:27.973599    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:28.920675 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:28.920643    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:29.349281 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:29.349245    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:29.704629 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:29.704530    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:30.119870 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:30.119834    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:32.421395 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:32.421360    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:32.780060 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:32.780022    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:32.881959 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:32.881921    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:33.364164 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:33.364102    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:34.110766 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.110733    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"]
Sep 18 19:06:34.111359 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.111343    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c0ebd9e-54bb-46a2-b81c-8f331512e0d6" containerName="maas-api"
Sep 18 19:06:34.111408 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.111362    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c0ebd9e-54bb-46a2-b81c-8f331512e0d6" containerName="maas-api"
Sep 18 19:06:34.111408 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.111389    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e1d1183-db92-40da-89f1-6b1d8302411f" containerName="maas-api"
Sep 18 19:06:34.111408 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.111398    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e1d1183-db92-40da-89f1-6b1d8302411f" containerName="maas-api"
Sep 18 19:06:34.111502 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.111413    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df43d446-cb9a-4201-96e9-f40f93b98955" containerName="payload-processing"
Sep 18 19:06:34.111502 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.111422    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="df43d446-cb9a-4201-96e9-f40f93b98955" containerName="payload-processing"
Sep 18 19:06:34.111502 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.111437    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3d18575-7648-42f0-a048-ce5fe38237e9" containerName="maas-api"
Sep 18 19:06:34.111502 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.111445    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3d18575-7648-42f0-a048-ce5fe38237e9" containerName="maas-api"
Sep 18 19:06:34.111502 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.111465    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4b2278c-a55a-43cf-9c36-528713c72556" containerName="test-curl-8953-62238a"
Sep 18 19:06:34.111502 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.111473    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4b2278c-a55a-43cf-9c36-528713c72556" containerName="test-curl-8953-62238a"
Sep 18 19:06:34.111502 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.111484    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac14ae1c-9d31-465c-8de0-a6ab51b5e00e" containerName="payload-pre-processing"
Sep 18 19:06:34.111502 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.111494    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac14ae1c-9d31-465c-8de0-a6ab51b5e00e" containerName="payload-pre-processing"
Sep 18 19:06:34.111739 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.111584    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c0ebd9e-54bb-46a2-b81c-8f331512e0d6" containerName="maas-api"
Sep 18 19:06:34.111739 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.111597    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e1d1183-db92-40da-89f1-6b1d8302411f" containerName="maas-api"
Sep 18 19:06:34.111739 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.111607    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac14ae1c-9d31-465c-8de0-a6ab51b5e00e" containerName="payload-pre-processing"
Sep 18 19:06:34.111739 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.111620    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="df43d446-cb9a-4201-96e9-f40f93b98955" containerName="payload-processing"
Sep 18 19:06:34.111739 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.111631    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3d18575-7648-42f0-a048-ce5fe38237e9" containerName="maas-api"
Sep 18 19:06:34.111739 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.111643    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4b2278c-a55a-43cf-9c36-528713c72556" containerName="test-curl-8953-62238a"
Sep 18 19:06:34.116932 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.116909    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:06:34.119546 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.119524    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-0991d8-serving-cert\""
Sep 18 19:06:34.127643 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.127621    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"]
Sep 18 19:06:34.147286 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.147260    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:34.164373 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.164350    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d5c077-92fb-4206-8360-4e6492eb45ae-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr\" (UID: \"a2d5c077-92fb-4206-8360-4e6492eb45ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:06:34.164464 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.164384    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqp8p\" (UniqueName: \"kubernetes.io/projected/a2d5c077-92fb-4206-8360-4e6492eb45ae-kube-api-access-tqp8p\") pod \"maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr\" (UID: \"a2d5c077-92fb-4206-8360-4e6492eb45ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:06:34.164464 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.164431    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d5c077-92fb-4206-8360-4e6492eb45ae-metrics-certs\") pod \"maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr\" (UID: \"a2d5c077-92fb-4206-8360-4e6492eb45ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:06:34.164540 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.164472    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d5c077-92fb-4206-8360-4e6492eb45ae-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr\" (UID: \"a2d5c077-92fb-4206-8360-4e6492eb45ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:06:34.219221 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.219190    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:34.265264 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.265242    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqp8p\" (UniqueName: \"kubernetes.io/projected/a2d5c077-92fb-4206-8360-4e6492eb45ae-kube-api-access-tqp8p\") pod \"maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr\" (UID: \"a2d5c077-92fb-4206-8360-4e6492eb45ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:06:34.265372 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.265299    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d5c077-92fb-4206-8360-4e6492eb45ae-metrics-certs\") pod \"maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr\" (UID: \"a2d5c077-92fb-4206-8360-4e6492eb45ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:06:34.265372 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.265343    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d5c077-92fb-4206-8360-4e6492eb45ae-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr\" (UID: \"a2d5c077-92fb-4206-8360-4e6492eb45ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:06:34.265499 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.265376    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d5c077-92fb-4206-8360-4e6492eb45ae-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr\" (UID: \"a2d5c077-92fb-4206-8360-4e6492eb45ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:06:34.265499 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:06:34.265469    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-serving-cert: secret "maas-api-e2e-ipp-a-0991d8-serving-cert" not found
Sep 18 19:06:34.265606 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:06:34.265533    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2d5c077-92fb-4206-8360-4e6492eb45ae-maas-api-tls podName:a2d5c077-92fb-4206-8360-4e6492eb45ae nodeName:}" failed. No retries permitted until 2026-09-18 19:06:34.765517439 +0000 UTC m=+1630.964001622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a2d5c077-92fb-4206-8360-4e6492eb45ae-maas-api-tls") pod "maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr" (UID: "a2d5c077-92fb-4206-8360-4e6492eb45ae") : secret "maas-api-e2e-ipp-a-0991d8-serving-cert" not found
Sep 18 19:06:34.265969 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.265940    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d5c077-92fb-4206-8360-4e6492eb45ae-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr\" (UID: \"a2d5c077-92fb-4206-8360-4e6492eb45ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:06:34.267735 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.267717    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d5c077-92fb-4206-8360-4e6492eb45ae-metrics-certs\") pod \"maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr\" (UID: \"a2d5c077-92fb-4206-8360-4e6492eb45ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:06:34.274470 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.274448    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqp8p\" (UniqueName: \"kubernetes.io/projected/a2d5c077-92fb-4206-8360-4e6492eb45ae-kube-api-access-tqp8p\") pod \"maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr\" (UID: \"a2d5c077-92fb-4206-8360-4e6492eb45ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:06:34.429356 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.429280    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:34.638205 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.638169    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:34.736864 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.736758    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:34.772120 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.772089    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d5c077-92fb-4206-8360-4e6492eb45ae-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr\" (UID: \"a2d5c077-92fb-4206-8360-4e6492eb45ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:06:34.774597 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.774577    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d5c077-92fb-4206-8360-4e6492eb45ae-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr\" (UID: \"a2d5c077-92fb-4206-8360-4e6492eb45ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:06:34.783309 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.783284    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5"]
Sep 18 19:06:34.787296 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.787279    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5"
Sep 18 19:06:34.790782 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.790762    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-0991d8\""
Sep 18 19:06:34.790897 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.790815    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-0991d8-dockercfg-jvjlt\""
Sep 18 19:06:34.798987 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.798965    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5"]
Sep 18 19:06:34.822509 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.822474    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47"]
Sep 18 19:06:34.825967 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.825948    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47"
Sep 18 19:06:34.840445 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.840424    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47"]
Sep 18 19:06:34.873401 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.873376    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdjzn\" (UniqueName: \"kubernetes.io/projected/b00d12fb-4364-47a0-89eb-2f68505abdc8-kube-api-access-xdjzn\") pod \"payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5\" (UID: \"b00d12fb-4364-47a0-89eb-2f68505abdc8\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5"
Sep 18 19:06:34.873540 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.873430    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79bfc37c-f7d4-4045-aec3-8bf8031901dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47\" (UID: \"79bfc37c-f7d4-4045-aec3-8bf8031901dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47"
Sep 18 19:06:34.873540 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.873477    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqzsv\" (UniqueName: \"kubernetes.io/projected/79bfc37c-f7d4-4045-aec3-8bf8031901dd-kube-api-access-kqzsv\") pod \"payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47\" (UID: \"79bfc37c-f7d4-4045-aec3-8bf8031901dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47"
Sep 18 19:06:34.873540 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.873498    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b00d12fb-4364-47a0-89eb-2f68505abdc8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5\" (UID: \"b00d12fb-4364-47a0-89eb-2f68505abdc8\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5"
Sep 18 19:06:34.974675 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.974649    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdjzn\" (UniqueName: \"kubernetes.io/projected/b00d12fb-4364-47a0-89eb-2f68505abdc8-kube-api-access-xdjzn\") pod \"payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5\" (UID: \"b00d12fb-4364-47a0-89eb-2f68505abdc8\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5"
Sep 18 19:06:34.974853 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.974687    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79bfc37c-f7d4-4045-aec3-8bf8031901dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47\" (UID: \"79bfc37c-f7d4-4045-aec3-8bf8031901dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47"
Sep 18 19:06:34.974853 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.974734    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqzsv\" (UniqueName: \"kubernetes.io/projected/79bfc37c-f7d4-4045-aec3-8bf8031901dd-kube-api-access-kqzsv\") pod \"payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47\" (UID: \"79bfc37c-f7d4-4045-aec3-8bf8031901dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47"
Sep 18 19:06:34.974853 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.974778    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b00d12fb-4364-47a0-89eb-2f68505abdc8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5\" (UID: \"b00d12fb-4364-47a0-89eb-2f68505abdc8\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5"
Sep 18 19:06:34.975558 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.975536    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79bfc37c-f7d4-4045-aec3-8bf8031901dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47\" (UID: \"79bfc37c-f7d4-4045-aec3-8bf8031901dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47"
Sep 18 19:06:34.975671 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.975635    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b00d12fb-4364-47a0-89eb-2f68505abdc8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5\" (UID: \"b00d12fb-4364-47a0-89eb-2f68505abdc8\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5"
Sep 18 19:06:34.982730 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.982702    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdjzn\" (UniqueName: \"kubernetes.io/projected/b00d12fb-4364-47a0-89eb-2f68505abdc8-kube-api-access-xdjzn\") pod \"payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5\" (UID: \"b00d12fb-4364-47a0-89eb-2f68505abdc8\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5"
Sep 18 19:06:34.982873 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:34.982845    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqzsv\" (UniqueName: \"kubernetes.io/projected/79bfc37c-f7d4-4045-aec3-8bf8031901dd-kube-api-access-kqzsv\") pod \"payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47\" (UID: \"79bfc37c-f7d4-4045-aec3-8bf8031901dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47"
Sep 18 19:06:35.028407 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:35.028376    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:06:35.098402 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:35.098378    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5"
Sep 18 19:06:35.135361 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:35.135319    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47"
Sep 18 19:06:35.302209 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:35.302187    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"]
Sep 18 19:06:35.303621 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:06:35.303593    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2d5c077_92fb_4206_8360_4e6492eb45ae.slice/crio-08721a11b77c353f5d0c953cad3b4ef596a8cc536237b70bf348b4f47af90477 WatchSource:0}: Error finding container 08721a11b77c353f5d0c953cad3b4ef596a8cc536237b70bf348b4f47af90477: Status 404 returned error can't find the container with id 08721a11b77c353f5d0c953cad3b4ef596a8cc536237b70bf348b4f47af90477
Sep 18 19:06:35.378506 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:35.378484    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5"]
Sep 18 19:06:35.379851 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:06:35.379804    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb00d12fb_4364_47a0_89eb_2f68505abdc8.slice/crio-d222f6382144235dc09304ad2cbff1ddbfceee90ceead75a7447840bc6e0a584 WatchSource:0}: Error finding container d222f6382144235dc09304ad2cbff1ddbfceee90ceead75a7447840bc6e0a584: Status 404 returned error can't find the container with id d222f6382144235dc09304ad2cbff1ddbfceee90ceead75a7447840bc6e0a584
Sep 18 19:06:35.414759 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:35.414737    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47"]
Sep 18 19:06:35.417936 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:06:35.417910    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79bfc37c_f7d4_4045_aec3_8bf8031901dd.slice/crio-1b09bd1b2a74a3eb7cba029728453bd371176c190a8f1b5445cefc6549a1ee8c WatchSource:0}: Error finding container 1b09bd1b2a74a3eb7cba029728453bd371176c190a8f1b5445cefc6549a1ee8c: Status 404 returned error can't find the container with id 1b09bd1b2a74a3eb7cba029728453bd371176c190a8f1b5445cefc6549a1ee8c
Sep 18 19:06:36.289437 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:36.289397    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr" event={"ID":"a2d5c077-92fb-4206-8360-4e6492eb45ae","Type":"ContainerStarted","Data":"bb9cc525811391446ae05519e844ffb5a7e1cea2355cd6add98624b1c30c789a"}
Sep 18 19:06:36.289910 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:36.289462    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:06:36.289910 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:36.289486    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr" event={"ID":"a2d5c077-92fb-4206-8360-4e6492eb45ae","Type":"ContainerStarted","Data":"08721a11b77c353f5d0c953cad3b4ef596a8cc536237b70bf348b4f47af90477"}
Sep 18 19:06:36.291066 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:36.291039    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47" event={"ID":"79bfc37c-f7d4-4045-aec3-8bf8031901dd","Type":"ContainerStarted","Data":"1414680dafd278b57b8abbeb462a457f50c1fc36f5d9f3beabfe23f46be98100"}
Sep 18 19:06:36.291162 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:36.291080    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47" event={"ID":"79bfc37c-f7d4-4045-aec3-8bf8031901dd","Type":"ContainerStarted","Data":"1b09bd1b2a74a3eb7cba029728453bd371176c190a8f1b5445cefc6549a1ee8c"}
Sep 18 19:06:36.291162 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:36.291111    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47"
Sep 18 19:06:36.292630 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:36.292612    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5" event={"ID":"b00d12fb-4364-47a0-89eb-2f68505abdc8","Type":"ContainerStarted","Data":"f57a10618169b4b88cbd576b6744fc7426d614f1f53f9b97622c6303e81c1f5f"}
Sep 18 19:06:36.292719 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:36.292651    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5" event={"ID":"b00d12fb-4364-47a0-89eb-2f68505abdc8","Type":"ContainerStarted","Data":"d222f6382144235dc09304ad2cbff1ddbfceee90ceead75a7447840bc6e0a584"}
Sep 18 19:06:36.292719 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:36.292675    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5"
Sep 18 19:06:36.306813 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:36.306771    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr" podStartSLOduration=1.83282221 podStartE2EDuration="2.306758338s" totalImagesPullingTime="473.936128ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:06:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:06:36.306314793 +0000 UTC m=+1632.504799013" watchObservedRunningTime="2026-09-18 19:06:36.306758338 +0000 UTC m=+1632.505242543"
Sep 18 19:06:36.323553 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:36.323507    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5" podStartSLOduration=2.323494805 podStartE2EDuration="2.323494805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:06:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:06:36.321736323 +0000 UTC m=+1632.520220551" watchObservedRunningTime="2026-09-18 19:06:36.323494805 +0000 UTC m=+1632.521979056"
Sep 18 19:06:36.340032 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:36.339987    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47" podStartSLOduration=2.339975064 podStartE2EDuration="2.339975064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:06:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:06:36.337642939 +0000 UTC m=+1632.536127163" watchObservedRunningTime="2026-09-18 19:06:36.339975064 +0000 UTC m=+1632.538459267"
Sep 18 19:06:42.303474 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:42.303436    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:06:47.300508 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:47.300477    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47"
Sep 18 19:06:47.300974 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:47.300541    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5"
Sep 18 19:06:56.248459 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:56.248419    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:56.925322 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:56.925291    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:57.493668 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:57.493636    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:57.753221 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:57.753131    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:58.420015 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:58.419981    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:58.543752 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:58.543717    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:58.830127 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:58.830090    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:06:59.839572 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:06:59.839535    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:01.324098 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:01.324061    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:01.992123 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:01.992087    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:02.110333 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:02.110300    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:02.265120 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:02.265039    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:03.362092 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:03.362032    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:04.311831 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:04.311775    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:05.151448 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.151413    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"]
Sep 18 19:07:05.155361 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.155335    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:07:05.158152 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.158130    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-d6246c-serving-cert\""
Sep 18 19:07:05.169168 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.169141    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"]
Sep 18 19:07:05.205969 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.205934    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:05.256638 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.256603    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4732967a-67fc-49a0-87f3-8e602993ec54-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln\" (UID: \"4732967a-67fc-49a0-87f3-8e602993ec54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:07:05.256801 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.256650    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4732967a-67fc-49a0-87f3-8e602993ec54-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln\" (UID: \"4732967a-67fc-49a0-87f3-8e602993ec54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:07:05.256801 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.256673    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kmpc\" (UniqueName: \"kubernetes.io/projected/4732967a-67fc-49a0-87f3-8e602993ec54-kube-api-access-5kmpc\") pod \"maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln\" (UID: \"4732967a-67fc-49a0-87f3-8e602993ec54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:07:05.256801 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.256715    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4732967a-67fc-49a0-87f3-8e602993ec54-metrics-certs\") pod \"maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln\" (UID: \"4732967a-67fc-49a0-87f3-8e602993ec54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:07:05.309830 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.309800    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:05.358022 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.357988    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4732967a-67fc-49a0-87f3-8e602993ec54-metrics-certs\") pod \"maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln\" (UID: \"4732967a-67fc-49a0-87f3-8e602993ec54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:07:05.358187 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.358070    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4732967a-67fc-49a0-87f3-8e602993ec54-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln\" (UID: \"4732967a-67fc-49a0-87f3-8e602993ec54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:07:05.358187 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.358125    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4732967a-67fc-49a0-87f3-8e602993ec54-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln\" (UID: \"4732967a-67fc-49a0-87f3-8e602993ec54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:07:05.358187 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.358159    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kmpc\" (UniqueName: \"kubernetes.io/projected/4732967a-67fc-49a0-87f3-8e602993ec54-kube-api-access-5kmpc\") pod \"maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln\" (UID: \"4732967a-67fc-49a0-87f3-8e602993ec54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:07:05.358360 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:07:05.358310    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-serving-cert: secret "maas-api-e2e-ipp-b-d6246c-serving-cert" not found
Sep 18 19:07:05.358413 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:07:05.358405    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4732967a-67fc-49a0-87f3-8e602993ec54-maas-api-tls podName:4732967a-67fc-49a0-87f3-8e602993ec54 nodeName:}" failed. No retries permitted until 2026-09-18 19:07:05.858384647 +0000 UTC m=+1662.056868847 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4732967a-67fc-49a0-87f3-8e602993ec54-maas-api-tls") pod "maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln" (UID: "4732967a-67fc-49a0-87f3-8e602993ec54") : secret "maas-api-e2e-ipp-b-d6246c-serving-cert" not found
Sep 18 19:07:05.358748 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.358730    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4732967a-67fc-49a0-87f3-8e602993ec54-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln\" (UID: \"4732967a-67fc-49a0-87f3-8e602993ec54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:07:05.360479 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.360458    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4732967a-67fc-49a0-87f3-8e602993ec54-metrics-certs\") pod \"maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln\" (UID: \"4732967a-67fc-49a0-87f3-8e602993ec54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:07:05.382346 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.382319    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kmpc\" (UniqueName: \"kubernetes.io/projected/4732967a-67fc-49a0-87f3-8e602993ec54-kube-api-access-5kmpc\") pod \"maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln\" (UID: \"4732967a-67fc-49a0-87f3-8e602993ec54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:07:05.754148 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.754113    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:05.853770 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.853735    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:05.863473 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.863446    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4732967a-67fc-49a0-87f3-8e602993ec54-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln\" (UID: \"4732967a-67fc-49a0-87f3-8e602993ec54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:07:05.865608 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.865585    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4732967a-67fc-49a0-87f3-8e602993ec54-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln\" (UID: \"4732967a-67fc-49a0-87f3-8e602993ec54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:07:05.912646 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.912615    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57"]
Sep 18 19:07:05.916442 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.916427    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57"
Sep 18 19:07:05.920369 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.920352    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-d6246c-dockercfg-wtpq9\""
Sep 18 19:07:05.921083 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.921068    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-d6246c\""
Sep 18 19:07:05.937677 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.937655    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57"]
Sep 18 19:07:05.964236 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.964210    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flk8g\" (UniqueName: \"kubernetes.io/projected/095736cf-0478-48c9-b0f1-bba353a00a42-kube-api-access-flk8g\") pod \"payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57\" (UID: \"095736cf-0478-48c9-b0f1-bba353a00a42\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57"
Sep 18 19:07:05.964372 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.964248    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/095736cf-0478-48c9-b0f1-bba353a00a42-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57\" (UID: \"095736cf-0478-48c9-b0f1-bba353a00a42\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57"
Sep 18 19:07:05.979613 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.979587    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:05.980277 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.980258    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg"]
Sep 18 19:07:05.984075 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.984060    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg"
Sep 18 19:07:05.999535 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:05.999515    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg"]
Sep 18 19:07:06.065224 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:06.065144    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d1261f7-766c-42fe-b061-98b4893c03d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg\" (UID: \"6d1261f7-766c-42fe-b061-98b4893c03d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg"
Sep 18 19:07:06.065224 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:06.065187    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbx5z\" (UniqueName: \"kubernetes.io/projected/6d1261f7-766c-42fe-b061-98b4893c03d0-kube-api-access-bbx5z\") pod \"payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg\" (UID: \"6d1261f7-766c-42fe-b061-98b4893c03d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg"
Sep 18 19:07:06.065454 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:06.065223    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flk8g\" (UniqueName: \"kubernetes.io/projected/095736cf-0478-48c9-b0f1-bba353a00a42-kube-api-access-flk8g\") pod \"payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57\" (UID: \"095736cf-0478-48c9-b0f1-bba353a00a42\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57"
Sep 18 19:07:06.065454 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:06.065271    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/095736cf-0478-48c9-b0f1-bba353a00a42-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57\" (UID: \"095736cf-0478-48c9-b0f1-bba353a00a42\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57"
Sep 18 19:07:06.066071 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:06.066052    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/095736cf-0478-48c9-b0f1-bba353a00a42-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57\" (UID: \"095736cf-0478-48c9-b0f1-bba353a00a42\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57"
Sep 18 19:07:06.068064 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:06.068044    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:07:06.074273 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:06.074253    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flk8g\" (UniqueName: \"kubernetes.io/projected/095736cf-0478-48c9-b0f1-bba353a00a42-kube-api-access-flk8g\") pod \"payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57\" (UID: \"095736cf-0478-48c9-b0f1-bba353a00a42\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57"
Sep 18 19:07:06.166917 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:06.166865    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d1261f7-766c-42fe-b061-98b4893c03d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg\" (UID: \"6d1261f7-766c-42fe-b061-98b4893c03d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg"
Sep 18 19:07:06.167364 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:06.166936    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbx5z\" (UniqueName: \"kubernetes.io/projected/6d1261f7-766c-42fe-b061-98b4893c03d0-kube-api-access-bbx5z\") pod \"payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg\" (UID: \"6d1261f7-766c-42fe-b061-98b4893c03d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg"
Sep 18 19:07:06.167699 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:06.167674    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d1261f7-766c-42fe-b061-98b4893c03d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg\" (UID: \"6d1261f7-766c-42fe-b061-98b4893c03d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg"
Sep 18 19:07:06.176524 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:06.176504    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbx5z\" (UniqueName: \"kubernetes.io/projected/6d1261f7-766c-42fe-b061-98b4893c03d0-kube-api-access-bbx5z\") pod \"payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg\" (UID: \"6d1261f7-766c-42fe-b061-98b4893c03d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg"
Sep 18 19:07:06.227353 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:06.227323    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57"
Sep 18 19:07:06.294206 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:06.294176    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg"
Sep 18 19:07:06.348519 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:06.348423    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"]
Sep 18 19:07:06.357618 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:07:06.357577    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4732967a_67fc_49a0_87f3_8e602993ec54.slice/crio-8541afc40b493e6d242932e861c93e35ea4d6dfaeedd3f1075e0d8f2020398f3 WatchSource:0}: Error finding container 8541afc40b493e6d242932e861c93e35ea4d6dfaeedd3f1075e0d8f2020398f3: Status 404 returned error can't find the container with id 8541afc40b493e6d242932e861c93e35ea4d6dfaeedd3f1075e0d8f2020398f3
Sep 18 19:07:06.444865 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:06.444832    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln" event={"ID":"4732967a-67fc-49a0-87f3-8e602993ec54","Type":"ContainerStarted","Data":"8541afc40b493e6d242932e861c93e35ea4d6dfaeedd3f1075e0d8f2020398f3"}
Sep 18 19:07:06.511926 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:06.511889    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57"]
Sep 18 19:07:06.520121 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:07:06.520068    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod095736cf_0478_48c9_b0f1_bba353a00a42.slice/crio-0e4b6beb316d9d08ef2797d669ab31ea79f7edc32528d7041b3edc1498478f19 WatchSource:0}: Error finding container 0e4b6beb316d9d08ef2797d669ab31ea79f7edc32528d7041b3edc1498478f19: Status 404 returned error can't find the container with id 0e4b6beb316d9d08ef2797d669ab31ea79f7edc32528d7041b3edc1498478f19
Sep 18 19:07:06.577230 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:06.577204    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg"]
Sep 18 19:07:06.579331 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:07:06.579305    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d1261f7_766c_42fe_b061_98b4893c03d0.slice/crio-66bff94248db5d69be98cee8642575b1cc085f5620d3af1a48f39962721178cc WatchSource:0}: Error finding container 66bff94248db5d69be98cee8642575b1cc085f5620d3af1a48f39962721178cc: Status 404 returned error can't find the container with id 66bff94248db5d69be98cee8642575b1cc085f5620d3af1a48f39962721178cc
Sep 18 19:07:07.453194 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:07.453157    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57" event={"ID":"095736cf-0478-48c9-b0f1-bba353a00a42","Type":"ContainerStarted","Data":"221d58b86dfa1d2f43cd302c02aff9061a591a570f3453d457094fa53035fb92"}
Sep 18 19:07:07.453653 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:07.453215    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57" event={"ID":"095736cf-0478-48c9-b0f1-bba353a00a42","Type":"ContainerStarted","Data":"0e4b6beb316d9d08ef2797d669ab31ea79f7edc32528d7041b3edc1498478f19"}
Sep 18 19:07:07.453653 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:07.453364    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57"
Sep 18 19:07:07.454813 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:07.454787    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg" event={"ID":"6d1261f7-766c-42fe-b061-98b4893c03d0","Type":"ContainerStarted","Data":"248ee681e306c359f2b8ee1cd46010601d5f4dfb78225beaa6a3d997f7dd4d33"}
Sep 18 19:07:07.454968 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:07.454842    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg" event={"ID":"6d1261f7-766c-42fe-b061-98b4893c03d0","Type":"ContainerStarted","Data":"66bff94248db5d69be98cee8642575b1cc085f5620d3af1a48f39962721178cc"}
Sep 18 19:07:07.454968 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:07.454871    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg"
Sep 18 19:07:07.456297 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:07.456278    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln" event={"ID":"4732967a-67fc-49a0-87f3-8e602993ec54","Type":"ContainerStarted","Data":"c04df5008d8f4aed4376d55ded592122dd807ea17c0dadaa49061daa844a485d"}
Sep 18 19:07:07.456400 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:07.456330    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:07:07.474553 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:07.474503    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57" podStartSLOduration=2.474486325 podStartE2EDuration="2.474486325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:07:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:07:07.471411384 +0000 UTC m=+1663.669895608" watchObservedRunningTime="2026-09-18 19:07:07.474486325 +0000 UTC m=+1663.672970530"
Sep 18 19:07:07.492779 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:07.492735    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln" podStartSLOduration=2.046866393 podStartE2EDuration="2.492719693s" totalImagesPullingTime="445.8533ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:07:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:07:07.492126916 +0000 UTC m=+1663.690611147" watchObservedRunningTime="2026-09-18 19:07:07.492719693 +0000 UTC m=+1663.691203898"
Sep 18 19:07:07.510811 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:07.510769    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg" podStartSLOduration=2.510758163 podStartE2EDuration="2.510758163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:07:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:07:07.509197506 +0000 UTC m=+1663.707681727" watchObservedRunningTime="2026-09-18 19:07:07.510758163 +0000 UTC m=+1663.709242368"
Sep 18 19:07:13.466919 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:13.466887    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:07:18.464425 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:18.464392    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57"
Sep 18 19:07:18.464855 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:18.464713    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg"
Sep 18 19:07:27.337812 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:27.337768    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:27.739664 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:27.739547    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:28.138529 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:28.138487    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:28.723391 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:28.723344    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:29.657444 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:29.657388    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:30.401009 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:30.400974    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:32.113976 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:32.113943    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:33.605250 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:33.605216    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:37.367291 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:37.367215    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57_095736cf-0478-48c9-b0f1-bba353a00a42/payload-processing/0.log"
Sep 18 19:07:48.819397 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:48.819363    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:48.919073 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:48.919032    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:07:49.031575 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:07:49.031530    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:00.881351 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:00.881311    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:23.660813 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:23.660781    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5_b00d12fb-4364-47a0-89eb-2f68505abdc8/payload-processing/0.log"
Sep 18 19:08:24.038912 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:24.038883    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57_095736cf-0478-48c9-b0f1-bba353a00a42/payload-processing/0.log"
Sep 18 19:08:24.155920 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:24.155891    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57_095736cf-0478-48c9-b0f1-bba353a00a42/payload-processing/0.log"
Sep 18 19:08:24.432730 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:24.432630    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:24.767780 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:24.767748    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:24.928070 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:24.928035    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:27.458350 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:27.458316    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:27.573039 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:27.573007    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:27.665151 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:27.665119    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:28.250480 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:28.250445    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:29.036327 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:29.036274    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:29.785347 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:29.785294    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:30.648611 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:30.648576    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:30.747060 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:30.747019    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:31.081299 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:31.081258    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:31.179871 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:31.179812    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:31.280099 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:31.280069    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:31.371716 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:31.371637    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94"]
Sep 18 19:08:31.375366 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:31.375351    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94"
Sep 18 19:08:31.378099 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:31.378081    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-5855d4-dockercfg-wpgmv\""
Sep 18 19:08:31.378524 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:31.378504    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-5855d4\""
Sep 18 19:08:31.385904 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:31.385882    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94"]
Sep 18 19:08:31.507660 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:31.507620    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0596fbfe-bf4a-4adc-ae55-70a47b2c748a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94\" (UID: \"0596fbfe-bf4a-4adc-ae55-70a47b2c748a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94"
Sep 18 19:08:31.507917 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:31.507783    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46mgh\" (UniqueName: \"kubernetes.io/projected/0596fbfe-bf4a-4adc-ae55-70a47b2c748a-kube-api-access-46mgh\") pod \"payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94\" (UID: \"0596fbfe-bf4a-4adc-ae55-70a47b2c748a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94"
Sep 18 19:08:31.608354 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:31.608319    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0596fbfe-bf4a-4adc-ae55-70a47b2c748a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94\" (UID: \"0596fbfe-bf4a-4adc-ae55-70a47b2c748a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94"
Sep 18 19:08:31.608517 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:31.608414    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46mgh\" (UniqueName: \"kubernetes.io/projected/0596fbfe-bf4a-4adc-ae55-70a47b2c748a-kube-api-access-46mgh\") pod \"payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94\" (UID: \"0596fbfe-bf4a-4adc-ae55-70a47b2c748a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94"
Sep 18 19:08:31.609079 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:31.609059    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0596fbfe-bf4a-4adc-ae55-70a47b2c748a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94\" (UID: \"0596fbfe-bf4a-4adc-ae55-70a47b2c748a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94"
Sep 18 19:08:31.617983 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:31.617922    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46mgh\" (UniqueName: \"kubernetes.io/projected/0596fbfe-bf4a-4adc-ae55-70a47b2c748a-kube-api-access-46mgh\") pod \"payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94\" (UID: \"0596fbfe-bf4a-4adc-ae55-70a47b2c748a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94"
Sep 18 19:08:31.687108 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:31.687029    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94"
Sep 18 19:08:31.974472 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:31.974443    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94"]
Sep 18 19:08:31.977538 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:08:31.977510    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0596fbfe_bf4a_4adc_ae55_70a47b2c748a.slice/crio-534f3ec6164fff2876fd76bc0991e1f29a436de4d58b195e81ef7e0ae7018c0b WatchSource:0}: Error finding container 534f3ec6164fff2876fd76bc0991e1f29a436de4d58b195e81ef7e0ae7018c0b: Status 404 returned error can't find the container with id 534f3ec6164fff2876fd76bc0991e1f29a436de4d58b195e81ef7e0ae7018c0b
Sep 18 19:08:32.913058 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:32.913019    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94" event={"ID":"0596fbfe-bf4a-4adc-ae55-70a47b2c748a","Type":"ContainerStarted","Data":"09b8f7d73fa622be35572a0b805fc75a3e41bbe51b38a06a9f8ee9642de9aef6"}
Sep 18 19:08:32.913465 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:32.913077    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94" event={"ID":"0596fbfe-bf4a-4adc-ae55-70a47b2c748a","Type":"ContainerStarted","Data":"534f3ec6164fff2876fd76bc0991e1f29a436de4d58b195e81ef7e0ae7018c0b"}
Sep 18 19:08:32.913465 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:32.913151    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94"
Sep 18 19:08:32.956579 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:32.956514    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94" podStartSLOduration=1.956492771 podStartE2EDuration="1.956492771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:08:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:08:32.953654696 +0000 UTC m=+1749.152138919" watchObservedRunningTime="2026-09-18 19:08:32.956492771 +0000 UTC m=+1749.154976977"
Sep 18 19:08:43.921004 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:43.920968    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94"
Sep 18 19:08:52.539419 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:52.539385    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:52.774367 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:52.774330    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:53.745124 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:53.745084    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:54.627003 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:54.626965    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:55.360063 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:55.360029    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:56.230061 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:56.230024    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:56.850255 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:56.850215    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:57.756515 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:57.756485    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:59.172420 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:59.172371    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:59.885042 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:59.885007    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:08:59.930326 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:59.930285    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94"]
Sep 18 19:08:59.930858 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:59.930562    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94" podUID="0596fbfe-bf4a-4adc-ae55-70a47b2c748a" containerName="payload-pre-processing" containerID="cri-o://09b8f7d73fa622be35572a0b805fc75a3e41bbe51b38a06a9f8ee9642de9aef6" gracePeriod=30
Sep 18 19:08:59.982700 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:08:59.982658    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:00.061188 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:00.061158    2558 generic.go:356] "Generic (PLEG): container finished" podID="0596fbfe-bf4a-4adc-ae55-70a47b2c748a" containerID="09b8f7d73fa622be35572a0b805fc75a3e41bbe51b38a06a9f8ee9642de9aef6" exitCode=0
Sep 18 19:09:00.061349 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:00.061233    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94" event={"ID":"0596fbfe-bf4a-4adc-ae55-70a47b2c748a","Type":"ContainerDied","Data":"09b8f7d73fa622be35572a0b805fc75a3e41bbe51b38a06a9f8ee9642de9aef6"}
Sep 18 19:09:00.089283 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:00.089228    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:00.186898 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:00.186874    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94"
Sep 18 19:09:00.190170 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:00.190140    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:00.279606 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:00.279571    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0596fbfe-bf4a-4adc-ae55-70a47b2c748a-plugins-config-volume\") pod \"0596fbfe-bf4a-4adc-ae55-70a47b2c748a\" (UID: \"0596fbfe-bf4a-4adc-ae55-70a47b2c748a\") "
Sep 18 19:09:00.279797 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:00.279618    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46mgh\" (UniqueName: \"kubernetes.io/projected/0596fbfe-bf4a-4adc-ae55-70a47b2c748a-kube-api-access-46mgh\") pod \"0596fbfe-bf4a-4adc-ae55-70a47b2c748a\" (UID: \"0596fbfe-bf4a-4adc-ae55-70a47b2c748a\") "
Sep 18 19:09:00.279968 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:00.279944    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0596fbfe-bf4a-4adc-ae55-70a47b2c748a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0596fbfe-bf4a-4adc-ae55-70a47b2c748a" (UID: "0596fbfe-bf4a-4adc-ae55-70a47b2c748a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:09:00.281768 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:00.281744    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0596fbfe-bf4a-4adc-ae55-70a47b2c748a-kube-api-access-46mgh" (OuterVolumeSpecName: "kube-api-access-46mgh") pod "0596fbfe-bf4a-4adc-ae55-70a47b2c748a" (UID: "0596fbfe-bf4a-4adc-ae55-70a47b2c748a"). InnerVolumeSpecName "kube-api-access-46mgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:09:00.380499 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:00.380462    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0596fbfe-bf4a-4adc-ae55-70a47b2c748a-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:09:00.380499 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:00.380496    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46mgh\" (UniqueName: \"kubernetes.io/projected/0596fbfe-bf4a-4adc-ae55-70a47b2c748a-kube-api-access-46mgh\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:09:00.515878 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:00.515847    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:00.681556 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:00.681518    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:01.068367 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:01.068340    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94"
Sep 18 19:09:01.068556 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:01.068373    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94" event={"ID":"0596fbfe-bf4a-4adc-ae55-70a47b2c748a","Type":"ContainerDied","Data":"534f3ec6164fff2876fd76bc0991e1f29a436de4d58b195e81ef7e0ae7018c0b"}
Sep 18 19:09:01.068556 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:01.068428    2558 scope.go:117] "RemoveContainer" containerID="09b8f7d73fa622be35572a0b805fc75a3e41bbe51b38a06a9f8ee9642de9aef6"
Sep 18 19:09:01.101046 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:01.101013    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94"]
Sep 18 19:09:01.104500 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:01.104474    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5855d4-d65c69958-jfl94"]
Sep 18 19:09:01.638235 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:01.638191    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:01.844904 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:01.844870    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0596fbfe-bf4a-4adc-ae55-70a47b2c748a" path="/var/lib/kubelet/pods/0596fbfe-bf4a-4adc-ae55-70a47b2c748a/volumes"
Sep 18 19:09:02.515621 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:02.515587    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:03.270938 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:03.270904    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:04.259289 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:04.259253    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:07.386491 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:07.386405    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:07.503075 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:07.503040    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:07.740123 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:07.740033    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:11.459900 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:11.459861    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:12.296137 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:12.296096    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:13.243846 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:13.243792    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:13.921471 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:13.921442    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:14.708935 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:14.708905    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:15.565336 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:15.565304    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:25.916099 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:25.916075    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bwsg_636f0979-fe2e-49fd-86d6-31bd7961e736/ovn-acl-logging/0.log"
Sep 18 19:09:25.923517 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:25.923497    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bwsg_636f0979-fe2e-49fd-86d6-31bd7961e736/ovn-acl-logging/0.log"
Sep 18 19:09:27.006362 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.006322    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"]
Sep 18 19:09:27.006800 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.006582    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr" podUID="a2d5c077-92fb-4206-8360-4e6492eb45ae" containerName="maas-api" containerID="cri-o://bb9cc525811391446ae05519e844ffb5a7e1cea2355cd6add98624b1c30c789a" gracePeriod=30
Sep 18 19:09:27.113708 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.113675    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:27.152056 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.152011    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5"]
Sep 18 19:09:27.152303 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.152225    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5" podUID="b00d12fb-4364-47a0-89eb-2f68505abdc8" containerName="payload-processing" containerID="cri-o://f57a10618169b4b88cbd576b6744fc7426d614f1f53f9b97622c6303e81c1f5f" gracePeriod=30
Sep 18 19:09:27.198734 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.198704    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47"]
Sep 18 19:09:27.199434 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.199389    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47" podUID="79bfc37c-f7d4-4045-aec3-8bf8031901dd" containerName="payload-pre-processing" containerID="cri-o://1414680dafd278b57b8abbeb462a457f50c1fc36f5d9f3beabfe23f46be98100" gracePeriod=30
Sep 18 19:09:27.216289 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.216259    2558 generic.go:356] "Generic (PLEG): container finished" podID="a2d5c077-92fb-4206-8360-4e6492eb45ae" containerID="bb9cc525811391446ae05519e844ffb5a7e1cea2355cd6add98624b1c30c789a" exitCode=0
Sep 18 19:09:27.216413 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.216329    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr" event={"ID":"a2d5c077-92fb-4206-8360-4e6492eb45ae","Type":"ContainerDied","Data":"bb9cc525811391446ae05519e844ffb5a7e1cea2355cd6add98624b1c30c789a"}
Sep 18 19:09:27.298495 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.298453    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47" podUID="79bfc37c-f7d4-4045-aec3-8bf8031901dd" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.79:9004: connect: connection refused"
Sep 18 19:09:27.298649 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.298453    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5" podUID="b00d12fb-4364-47a0-89eb-2f68505abdc8" containerName="payload-processing" probeResult="failure" output="dial tcp 10.134.0.78:9004: connect: connection refused"
Sep 18 19:09:27.315984 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.315948    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:27.330777 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.328087    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:09:27.338468 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.338155    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d5c077-92fb-4206-8360-4e6492eb45ae-openshift-service-ca\") pod \"a2d5c077-92fb-4206-8360-4e6492eb45ae\" (UID: \"a2d5c077-92fb-4206-8360-4e6492eb45ae\") "
Sep 18 19:09:27.338468 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.338311    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqp8p\" (UniqueName: \"kubernetes.io/projected/a2d5c077-92fb-4206-8360-4e6492eb45ae-kube-api-access-tqp8p\") pod \"a2d5c077-92fb-4206-8360-4e6492eb45ae\" (UID: \"a2d5c077-92fb-4206-8360-4e6492eb45ae\") "
Sep 18 19:09:27.338468 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.338402    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d5c077-92fb-4206-8360-4e6492eb45ae-metrics-certs\") pod \"a2d5c077-92fb-4206-8360-4e6492eb45ae\" (UID: \"a2d5c077-92fb-4206-8360-4e6492eb45ae\") "
Sep 18 19:09:27.339089 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.338866    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2d5c077-92fb-4206-8360-4e6492eb45ae-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a2d5c077-92fb-4206-8360-4e6492eb45ae" (UID: "a2d5c077-92fb-4206-8360-4e6492eb45ae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:09:27.341961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.341020    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d5c077-92fb-4206-8360-4e6492eb45ae-maas-api-tls\") pod \"a2d5c077-92fb-4206-8360-4e6492eb45ae\" (UID: \"a2d5c077-92fb-4206-8360-4e6492eb45ae\") "
Sep 18 19:09:27.341961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.341387    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d5c077-92fb-4206-8360-4e6492eb45ae-openshift-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:09:27.341961 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.341884    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2d5c077-92fb-4206-8360-4e6492eb45ae-kube-api-access-tqp8p" (OuterVolumeSpecName: "kube-api-access-tqp8p") pod "a2d5c077-92fb-4206-8360-4e6492eb45ae" (UID: "a2d5c077-92fb-4206-8360-4e6492eb45ae"). InnerVolumeSpecName "kube-api-access-tqp8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:09:27.343731 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.343699    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2d5c077-92fb-4206-8360-4e6492eb45ae-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a2d5c077-92fb-4206-8360-4e6492eb45ae" (UID: "a2d5c077-92fb-4206-8360-4e6492eb45ae"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:09:27.345479 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.345384    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2d5c077-92fb-4206-8360-4e6492eb45ae-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a2d5c077-92fb-4206-8360-4e6492eb45ae" (UID: "a2d5c077-92fb-4206-8360-4e6492eb45ae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:09:27.443494 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.443451    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqp8p\" (UniqueName: \"kubernetes.io/projected/a2d5c077-92fb-4206-8360-4e6492eb45ae-kube-api-access-tqp8p\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:09:27.443494 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.443494    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d5c077-92fb-4206-8360-4e6492eb45ae-metrics-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:09:27.443679 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.443510    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d5c077-92fb-4206-8360-4e6492eb45ae-maas-api-tls\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:09:27.477493 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.477469    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5"
Sep 18 19:09:27.482411 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.482385    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47"
Sep 18 19:09:27.544264 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.544237    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqzsv\" (UniqueName: \"kubernetes.io/projected/79bfc37c-f7d4-4045-aec3-8bf8031901dd-kube-api-access-kqzsv\") pod \"79bfc37c-f7d4-4045-aec3-8bf8031901dd\" (UID: \"79bfc37c-f7d4-4045-aec3-8bf8031901dd\") "
Sep 18 19:09:27.544419 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.544275    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b00d12fb-4364-47a0-89eb-2f68505abdc8-plugins-config-volume\") pod \"b00d12fb-4364-47a0-89eb-2f68505abdc8\" (UID: \"b00d12fb-4364-47a0-89eb-2f68505abdc8\") "
Sep 18 19:09:27.544419 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.544349    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79bfc37c-f7d4-4045-aec3-8bf8031901dd-plugins-config-volume\") pod \"79bfc37c-f7d4-4045-aec3-8bf8031901dd\" (UID: \"79bfc37c-f7d4-4045-aec3-8bf8031901dd\") "
Sep 18 19:09:27.544520 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.544424    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdjzn\" (UniqueName: \"kubernetes.io/projected/b00d12fb-4364-47a0-89eb-2f68505abdc8-kube-api-access-xdjzn\") pod \"b00d12fb-4364-47a0-89eb-2f68505abdc8\" (UID: \"b00d12fb-4364-47a0-89eb-2f68505abdc8\") "
Sep 18 19:09:27.544788 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.544749    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b00d12fb-4364-47a0-89eb-2f68505abdc8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b00d12fb-4364-47a0-89eb-2f68505abdc8" (UID: "b00d12fb-4364-47a0-89eb-2f68505abdc8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:09:27.544788 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.544775    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79bfc37c-f7d4-4045-aec3-8bf8031901dd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "79bfc37c-f7d4-4045-aec3-8bf8031901dd" (UID: "79bfc37c-f7d4-4045-aec3-8bf8031901dd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:09:27.546531 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.546507    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79bfc37c-f7d4-4045-aec3-8bf8031901dd-kube-api-access-kqzsv" (OuterVolumeSpecName: "kube-api-access-kqzsv") pod "79bfc37c-f7d4-4045-aec3-8bf8031901dd" (UID: "79bfc37c-f7d4-4045-aec3-8bf8031901dd"). InnerVolumeSpecName "kube-api-access-kqzsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:09:27.546618 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.546544    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b00d12fb-4364-47a0-89eb-2f68505abdc8-kube-api-access-xdjzn" (OuterVolumeSpecName: "kube-api-access-xdjzn") pod "b00d12fb-4364-47a0-89eb-2f68505abdc8" (UID: "b00d12fb-4364-47a0-89eb-2f68505abdc8"). InnerVolumeSpecName "kube-api-access-xdjzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:09:27.559963 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.559933    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:27.646166 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.646081    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqzsv\" (UniqueName: \"kubernetes.io/projected/79bfc37c-f7d4-4045-aec3-8bf8031901dd-kube-api-access-kqzsv\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:09:27.646166 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.646113    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b00d12fb-4364-47a0-89eb-2f68505abdc8-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:09:27.646166 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.646124    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79bfc37c-f7d4-4045-aec3-8bf8031901dd-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:09:27.646166 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.646132    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdjzn\" (UniqueName: \"kubernetes.io/projected/b00d12fb-4364-47a0-89eb-2f68505abdc8-kube-api-access-xdjzn\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:09:27.695589 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:27.695534    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:28.161827 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.161763    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:28.222525 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.222495    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"
Sep 18 19:09:28.222525 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.222518    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr" event={"ID":"a2d5c077-92fb-4206-8360-4e6492eb45ae","Type":"ContainerDied","Data":"08721a11b77c353f5d0c953cad3b4ef596a8cc536237b70bf348b4f47af90477"}
Sep 18 19:09:28.222750 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.222579    2558 scope.go:117] "RemoveContainer" containerID="bb9cc525811391446ae05519e844ffb5a7e1cea2355cd6add98624b1c30c789a"
Sep 18 19:09:28.223905 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.223887    2558 generic.go:356] "Generic (PLEG): container finished" podID="79bfc37c-f7d4-4045-aec3-8bf8031901dd" containerID="1414680dafd278b57b8abbeb462a457f50c1fc36f5d9f3beabfe23f46be98100" exitCode=0
Sep 18 19:09:28.223990 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.223944    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47"
Sep 18 19:09:28.223990 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.223977    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47" event={"ID":"79bfc37c-f7d4-4045-aec3-8bf8031901dd","Type":"ContainerDied","Data":"1414680dafd278b57b8abbeb462a457f50c1fc36f5d9f3beabfe23f46be98100"}
Sep 18 19:09:28.224096 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.224014    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47" event={"ID":"79bfc37c-f7d4-4045-aec3-8bf8031901dd","Type":"ContainerDied","Data":"1b09bd1b2a74a3eb7cba029728453bd371176c190a8f1b5445cefc6549a1ee8c"}
Sep 18 19:09:28.225647 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.225624    2558 generic.go:356] "Generic (PLEG): container finished" podID="b00d12fb-4364-47a0-89eb-2f68505abdc8" containerID="f57a10618169b4b88cbd576b6744fc7426d614f1f53f9b97622c6303e81c1f5f" exitCode=0
Sep 18 19:09:28.225763 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.225665    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5"
Sep 18 19:09:28.225763 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.225711    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5" event={"ID":"b00d12fb-4364-47a0-89eb-2f68505abdc8","Type":"ContainerDied","Data":"f57a10618169b4b88cbd576b6744fc7426d614f1f53f9b97622c6303e81c1f5f"}
Sep 18 19:09:28.225930 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.225766    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5" event={"ID":"b00d12fb-4364-47a0-89eb-2f68505abdc8","Type":"ContainerDied","Data":"d222f6382144235dc09304ad2cbff1ddbfceee90ceead75a7447840bc6e0a584"}
Sep 18 19:09:28.232562 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.232548    2558 scope.go:117] "RemoveContainer" containerID="1414680dafd278b57b8abbeb462a457f50c1fc36f5d9f3beabfe23f46be98100"
Sep 18 19:09:28.241306 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.241289    2558 scope.go:117] "RemoveContainer" containerID="1414680dafd278b57b8abbeb462a457f50c1fc36f5d9f3beabfe23f46be98100"
Sep 18 19:09:28.241605 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:09:28.241586    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1414680dafd278b57b8abbeb462a457f50c1fc36f5d9f3beabfe23f46be98100\": container with ID starting with 1414680dafd278b57b8abbeb462a457f50c1fc36f5d9f3beabfe23f46be98100 not found: ID does not exist" containerID="1414680dafd278b57b8abbeb462a457f50c1fc36f5d9f3beabfe23f46be98100"
Sep 18 19:09:28.241666 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.241615    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1414680dafd278b57b8abbeb462a457f50c1fc36f5d9f3beabfe23f46be98100"} err="failed to get container status \"1414680dafd278b57b8abbeb462a457f50c1fc36f5d9f3beabfe23f46be98100\": rpc error: code = NotFound desc = could not find container \"1414680dafd278b57b8abbeb462a457f50c1fc36f5d9f3beabfe23f46be98100\": container with ID starting with 1414680dafd278b57b8abbeb462a457f50c1fc36f5d9f3beabfe23f46be98100 not found: ID does not exist"
Sep 18 19:09:28.241666 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.241633    2558 scope.go:117] "RemoveContainer" containerID="f57a10618169b4b88cbd576b6744fc7426d614f1f53f9b97622c6303e81c1f5f"
Sep 18 19:09:28.249306 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.249286    2558 scope.go:117] "RemoveContainer" containerID="f57a10618169b4b88cbd576b6744fc7426d614f1f53f9b97622c6303e81c1f5f"
Sep 18 19:09:28.249561 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:09:28.249542    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f57a10618169b4b88cbd576b6744fc7426d614f1f53f9b97622c6303e81c1f5f\": container with ID starting with f57a10618169b4b88cbd576b6744fc7426d614f1f53f9b97622c6303e81c1f5f not found: ID does not exist" containerID="f57a10618169b4b88cbd576b6744fc7426d614f1f53f9b97622c6303e81c1f5f"
Sep 18 19:09:28.249613 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.249567    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f57a10618169b4b88cbd576b6744fc7426d614f1f53f9b97622c6303e81c1f5f"} err="failed to get container status \"f57a10618169b4b88cbd576b6744fc7426d614f1f53f9b97622c6303e81c1f5f\": rpc error: code = NotFound desc = could not find container \"f57a10618169b4b88cbd576b6744fc7426d614f1f53f9b97622c6303e81c1f5f\": container with ID starting with f57a10618169b4b88cbd576b6744fc7426d614f1f53f9b97622c6303e81c1f5f not found: ID does not exist"
Sep 18 19:09:28.257087 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.257067    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"]
Sep 18 19:09:28.267182 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.267162    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr"]
Sep 18 19:09:28.290580 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.290528    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5"]
Sep 18 19:09:28.300052 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.300025    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0991d8-86f744cb4c-km8j5"]
Sep 18 19:09:28.331339 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.331314    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47"]
Sep 18 19:09:28.333638 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.333615    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-0991d8-b6bcc5d54-2fg47"]
Sep 18 19:09:28.465117 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:28.465079    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:29.228937 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:29.228901    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:29.845083 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:29.845056    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79bfc37c-f7d4-4045-aec3-8bf8031901dd" path="/var/lib/kubelet/pods/79bfc37c-f7d4-4045-aec3-8bf8031901dd/volumes"
Sep 18 19:09:29.845467 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:29.845453    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2d5c077-92fb-4206-8360-4e6492eb45ae" path="/var/lib/kubelet/pods/a2d5c077-92fb-4206-8360-4e6492eb45ae/volumes"
Sep 18 19:09:29.845786 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:29.845775    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b00d12fb-4364-47a0-89eb-2f68505abdc8" path="/var/lib/kubelet/pods/b00d12fb-4364-47a0-89eb-2f68505abdc8/volumes"
Sep 18 19:09:30.298657 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:30.298606    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0991d8-75cf6bd67d-7hgcr" podUID="a2d5c077-92fb-4206-8360-4e6492eb45ae" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.77:8443/health\": context deadline exceeded"
Sep 18 19:09:30.687367 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:30.687288    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:31.865136 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:31.865104    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:33.205747 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:33.205712    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:09:41.893264 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:09:41.893230    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:09.441064 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:09.441031    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:11.155394 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:11.155359    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:16.181301 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:16.181263    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:16.256601 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:16.256561    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:16.471594 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:16.471494    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:17.078248 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:17.078194    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:17.946346 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:17.946307    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:18.620393 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:18.620355    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:19.413292 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:19.413254    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:20.084264 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:20.084221    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:27.454700 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.454668    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"]
Sep 18 19:10:27.455149 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.454899    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln" podUID="4732967a-67fc-49a0-87f3-8e602993ec54" containerName="maas-api" containerID="cri-o://c04df5008d8f4aed4376d55ded592122dd807ea17c0dadaa49061daa844a485d" gracePeriod=30
Sep 18 19:10:27.497132 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.497101    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:27.519716 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.519676    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57"]
Sep 18 19:10:27.519969 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.519936    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57" podUID="095736cf-0478-48c9-b0f1-bba353a00a42" containerName="payload-processing" containerID="cri-o://221d58b86dfa1d2f43cd302c02aff9061a591a570f3453d457094fa53035fb92" gracePeriod=30
Sep 18 19:10:27.538079 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.534915    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg"]
Sep 18 19:10:27.538079 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.535233    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg" podUID="6d1261f7-766c-42fe-b061-98b4893c03d0" containerName="payload-pre-processing" containerID="cri-o://248ee681e306c359f2b8ee1cd46010601d5f4dfb78225beaa6a3d997f7dd4d33" gracePeriod=30
Sep 18 19:10:27.591200 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.591051    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:27.694008 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.693877    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:27.793923 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.793887    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:27.827878 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.827856    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57"
Sep 18 19:10:27.830706 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.830684    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:10:27.855228 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.855194    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg"
Sep 18 19:10:27.922701 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.922673    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d1261f7-766c-42fe-b061-98b4893c03d0-plugins-config-volume\") pod \"6d1261f7-766c-42fe-b061-98b4893c03d0\" (UID: \"6d1261f7-766c-42fe-b061-98b4893c03d0\") "
Sep 18 19:10:27.922878 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.922718    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4732967a-67fc-49a0-87f3-8e602993ec54-metrics-certs\") pod \"4732967a-67fc-49a0-87f3-8e602993ec54\" (UID: \"4732967a-67fc-49a0-87f3-8e602993ec54\") "
Sep 18 19:10:27.922878 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.922775    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbx5z\" (UniqueName: \"kubernetes.io/projected/6d1261f7-766c-42fe-b061-98b4893c03d0-kube-api-access-bbx5z\") pod \"6d1261f7-766c-42fe-b061-98b4893c03d0\" (UID: \"6d1261f7-766c-42fe-b061-98b4893c03d0\") "
Sep 18 19:10:27.922878 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.922800    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4732967a-67fc-49a0-87f3-8e602993ec54-openshift-service-ca\") pod \"4732967a-67fc-49a0-87f3-8e602993ec54\" (UID: \"4732967a-67fc-49a0-87f3-8e602993ec54\") "
Sep 18 19:10:27.922878 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.922835    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flk8g\" (UniqueName: \"kubernetes.io/projected/095736cf-0478-48c9-b0f1-bba353a00a42-kube-api-access-flk8g\") pod \"095736cf-0478-48c9-b0f1-bba353a00a42\" (UID: \"095736cf-0478-48c9-b0f1-bba353a00a42\") "
Sep 18 19:10:27.922878 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.922862    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4732967a-67fc-49a0-87f3-8e602993ec54-maas-api-tls\") pod \"4732967a-67fc-49a0-87f3-8e602993ec54\" (UID: \"4732967a-67fc-49a0-87f3-8e602993ec54\") "
Sep 18 19:10:27.923177 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.922889    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kmpc\" (UniqueName: \"kubernetes.io/projected/4732967a-67fc-49a0-87f3-8e602993ec54-kube-api-access-5kmpc\") pod \"4732967a-67fc-49a0-87f3-8e602993ec54\" (UID: \"4732967a-67fc-49a0-87f3-8e602993ec54\") "
Sep 18 19:10:27.923177 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.923135    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d1261f7-766c-42fe-b061-98b4893c03d0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6d1261f7-766c-42fe-b061-98b4893c03d0" (UID: "6d1261f7-766c-42fe-b061-98b4893c03d0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:10:27.923298 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.923204    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/095736cf-0478-48c9-b0f1-bba353a00a42-plugins-config-volume\") pod \"095736cf-0478-48c9-b0f1-bba353a00a42\" (UID: \"095736cf-0478-48c9-b0f1-bba353a00a42\") "
Sep 18 19:10:27.923298 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.923214    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4732967a-67fc-49a0-87f3-8e602993ec54-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4732967a-67fc-49a0-87f3-8e602993ec54" (UID: "4732967a-67fc-49a0-87f3-8e602993ec54"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:10:27.924343 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.923684    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/095736cf-0478-48c9-b0f1-bba353a00a42-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "095736cf-0478-48c9-b0f1-bba353a00a42" (UID: "095736cf-0478-48c9-b0f1-bba353a00a42"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:10:27.924343 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.923967    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4732967a-67fc-49a0-87f3-8e602993ec54-openshift-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:10:27.924343 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.923988    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/095736cf-0478-48c9-b0f1-bba353a00a42-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:10:27.924343 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.924017    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d1261f7-766c-42fe-b061-98b4893c03d0-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:10:27.925307 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.925285    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/095736cf-0478-48c9-b0f1-bba353a00a42-kube-api-access-flk8g" (OuterVolumeSpecName: "kube-api-access-flk8g") pod "095736cf-0478-48c9-b0f1-bba353a00a42" (UID: "095736cf-0478-48c9-b0f1-bba353a00a42"). InnerVolumeSpecName "kube-api-access-flk8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:10:27.925430 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.925327    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d1261f7-766c-42fe-b061-98b4893c03d0-kube-api-access-bbx5z" (OuterVolumeSpecName: "kube-api-access-bbx5z") pod "6d1261f7-766c-42fe-b061-98b4893c03d0" (UID: "6d1261f7-766c-42fe-b061-98b4893c03d0"). InnerVolumeSpecName "kube-api-access-bbx5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:10:27.925430 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.925403    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4732967a-67fc-49a0-87f3-8e602993ec54-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4732967a-67fc-49a0-87f3-8e602993ec54" (UID: "4732967a-67fc-49a0-87f3-8e602993ec54"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:10:27.925430 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.925417    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4732967a-67fc-49a0-87f3-8e602993ec54-kube-api-access-5kmpc" (OuterVolumeSpecName: "kube-api-access-5kmpc") pod "4732967a-67fc-49a0-87f3-8e602993ec54" (UID: "4732967a-67fc-49a0-87f3-8e602993ec54"). InnerVolumeSpecName "kube-api-access-5kmpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:10:27.925652 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:27.925638    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4732967a-67fc-49a0-87f3-8e602993ec54-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4732967a-67fc-49a0-87f3-8e602993ec54" (UID: "4732967a-67fc-49a0-87f3-8e602993ec54"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:10:28.024916 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.024888    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4732967a-67fc-49a0-87f3-8e602993ec54-metrics-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:10:28.024916 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.024917    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbx5z\" (UniqueName: \"kubernetes.io/projected/6d1261f7-766c-42fe-b061-98b4893c03d0-kube-api-access-bbx5z\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:10:28.025112 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.024928    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flk8g\" (UniqueName: \"kubernetes.io/projected/095736cf-0478-48c9-b0f1-bba353a00a42-kube-api-access-flk8g\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:10:28.025112 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.024938    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4732967a-67fc-49a0-87f3-8e602993ec54-maas-api-tls\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:10:28.025112 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.024946    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kmpc\" (UniqueName: \"kubernetes.io/projected/4732967a-67fc-49a0-87f3-8e602993ec54-kube-api-access-5kmpc\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:10:28.150725 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.150689    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:28.254212 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.254176    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:28.549160 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.549131    2558 generic.go:356] "Generic (PLEG): container finished" podID="4732967a-67fc-49a0-87f3-8e602993ec54" containerID="c04df5008d8f4aed4376d55ded592122dd807ea17c0dadaa49061daa844a485d" exitCode=0
Sep 18 19:10:28.549594 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.549178    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"
Sep 18 19:10:28.549594 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.549225    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln" event={"ID":"4732967a-67fc-49a0-87f3-8e602993ec54","Type":"ContainerDied","Data":"c04df5008d8f4aed4376d55ded592122dd807ea17c0dadaa49061daa844a485d"}
Sep 18 19:10:28.549594 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.549279    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln" event={"ID":"4732967a-67fc-49a0-87f3-8e602993ec54","Type":"ContainerDied","Data":"8541afc40b493e6d242932e861c93e35ea4d6dfaeedd3f1075e0d8f2020398f3"}
Sep 18 19:10:28.549594 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.549311    2558 scope.go:117] "RemoveContainer" containerID="c04df5008d8f4aed4376d55ded592122dd807ea17c0dadaa49061daa844a485d"
Sep 18 19:10:28.550946 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.550923    2558 generic.go:356] "Generic (PLEG): container finished" podID="095736cf-0478-48c9-b0f1-bba353a00a42" containerID="221d58b86dfa1d2f43cd302c02aff9061a591a570f3453d457094fa53035fb92" exitCode=0
Sep 18 19:10:28.551062 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.550965    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57"
Sep 18 19:10:28.551062 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.551017    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57" event={"ID":"095736cf-0478-48c9-b0f1-bba353a00a42","Type":"ContainerDied","Data":"221d58b86dfa1d2f43cd302c02aff9061a591a570f3453d457094fa53035fb92"}
Sep 18 19:10:28.551177 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.551102    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57" event={"ID":"095736cf-0478-48c9-b0f1-bba353a00a42","Type":"ContainerDied","Data":"0e4b6beb316d9d08ef2797d669ab31ea79f7edc32528d7041b3edc1498478f19"}
Sep 18 19:10:28.552601 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.552576    2558 generic.go:356] "Generic (PLEG): container finished" podID="6d1261f7-766c-42fe-b061-98b4893c03d0" containerID="248ee681e306c359f2b8ee1cd46010601d5f4dfb78225beaa6a3d997f7dd4d33" exitCode=0
Sep 18 19:10:28.552715 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.552636    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg" event={"ID":"6d1261f7-766c-42fe-b061-98b4893c03d0","Type":"ContainerDied","Data":"248ee681e306c359f2b8ee1cd46010601d5f4dfb78225beaa6a3d997f7dd4d33"}
Sep 18 19:10:28.552715 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.552651    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg"
Sep 18 19:10:28.552715 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.552676    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg" event={"ID":"6d1261f7-766c-42fe-b061-98b4893c03d0","Type":"ContainerDied","Data":"66bff94248db5d69be98cee8642575b1cc085f5620d3af1a48f39962721178cc"}
Sep 18 19:10:28.560174 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.560155    2558 scope.go:117] "RemoveContainer" containerID="c04df5008d8f4aed4376d55ded592122dd807ea17c0dadaa49061daa844a485d"
Sep 18 19:10:28.560509 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:10:28.560485    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c04df5008d8f4aed4376d55ded592122dd807ea17c0dadaa49061daa844a485d\": container with ID starting with c04df5008d8f4aed4376d55ded592122dd807ea17c0dadaa49061daa844a485d not found: ID does not exist" containerID="c04df5008d8f4aed4376d55ded592122dd807ea17c0dadaa49061daa844a485d"
Sep 18 19:10:28.560626 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.560512    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c04df5008d8f4aed4376d55ded592122dd807ea17c0dadaa49061daa844a485d"} err="failed to get container status \"c04df5008d8f4aed4376d55ded592122dd807ea17c0dadaa49061daa844a485d\": rpc error: code = NotFound desc = could not find container \"c04df5008d8f4aed4376d55ded592122dd807ea17c0dadaa49061daa844a485d\": container with ID starting with c04df5008d8f4aed4376d55ded592122dd807ea17c0dadaa49061daa844a485d not found: ID does not exist"
Sep 18 19:10:28.560626 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.560528    2558 scope.go:117] "RemoveContainer" containerID="221d58b86dfa1d2f43cd302c02aff9061a591a570f3453d457094fa53035fb92"
Sep 18 19:10:28.569640 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.569624    2558 scope.go:117] "RemoveContainer" containerID="221d58b86dfa1d2f43cd302c02aff9061a591a570f3453d457094fa53035fb92"
Sep 18 19:10:28.569971 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:10:28.569948    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"221d58b86dfa1d2f43cd302c02aff9061a591a570f3453d457094fa53035fb92\": container with ID starting with 221d58b86dfa1d2f43cd302c02aff9061a591a570f3453d457094fa53035fb92 not found: ID does not exist" containerID="221d58b86dfa1d2f43cd302c02aff9061a591a570f3453d457094fa53035fb92"
Sep 18 19:10:28.570052 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.569978    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"221d58b86dfa1d2f43cd302c02aff9061a591a570f3453d457094fa53035fb92"} err="failed to get container status \"221d58b86dfa1d2f43cd302c02aff9061a591a570f3453d457094fa53035fb92\": rpc error: code = NotFound desc = could not find container \"221d58b86dfa1d2f43cd302c02aff9061a591a570f3453d457094fa53035fb92\": container with ID starting with 221d58b86dfa1d2f43cd302c02aff9061a591a570f3453d457094fa53035fb92 not found: ID does not exist"
Sep 18 19:10:28.570052 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.569998    2558 scope.go:117] "RemoveContainer" containerID="248ee681e306c359f2b8ee1cd46010601d5f4dfb78225beaa6a3d997f7dd4d33"
Sep 18 19:10:28.580368 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.580352    2558 scope.go:117] "RemoveContainer" containerID="248ee681e306c359f2b8ee1cd46010601d5f4dfb78225beaa6a3d997f7dd4d33"
Sep 18 19:10:28.580666 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:10:28.580646    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"248ee681e306c359f2b8ee1cd46010601d5f4dfb78225beaa6a3d997f7dd4d33\": container with ID starting with 248ee681e306c359f2b8ee1cd46010601d5f4dfb78225beaa6a3d997f7dd4d33 not found: ID does not exist" containerID="248ee681e306c359f2b8ee1cd46010601d5f4dfb78225beaa6a3d997f7dd4d33"
Sep 18 19:10:28.580714 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.580672    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"248ee681e306c359f2b8ee1cd46010601d5f4dfb78225beaa6a3d997f7dd4d33"} err="failed to get container status \"248ee681e306c359f2b8ee1cd46010601d5f4dfb78225beaa6a3d997f7dd4d33\": rpc error: code = NotFound desc = could not find container \"248ee681e306c359f2b8ee1cd46010601d5f4dfb78225beaa6a3d997f7dd4d33\": container with ID starting with 248ee681e306c359f2b8ee1cd46010601d5f4dfb78225beaa6a3d997f7dd4d33 not found: ID does not exist"
Sep 18 19:10:28.580905 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.580893    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"]
Sep 18 19:10:28.582672 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.582652    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d6246c-96f85bfcc-nvvln"]
Sep 18 19:10:28.594214 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.594193    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57"]
Sep 18 19:10:28.598959 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.598939    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d6246c-567bd89dfc-z4l57"]
Sep 18 19:10:28.610454 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.610434    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg"]
Sep 18 19:10:28.613296 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.613276    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d6246c-5cd5c8fcd6-jctxg"]
Sep 18 19:10:28.794671 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:28.794640    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:29.547338 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:29.547304    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:29.845092 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:29.845000    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="095736cf-0478-48c9-b0f1-bba353a00a42" path="/var/lib/kubelet/pods/095736cf-0478-48c9-b0f1-bba353a00a42/volumes"
Sep 18 19:10:29.845597 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:29.845575    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4732967a-67fc-49a0-87f3-8e602993ec54" path="/var/lib/kubelet/pods/4732967a-67fc-49a0-87f3-8e602993ec54/volumes"
Sep 18 19:10:29.846001 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:29.845981    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d1261f7-766c-42fe-b061-98b4893c03d0" path="/var/lib/kubelet/pods/6d1261f7-766c-42fe-b061-98b4893c03d0/volumes"
Sep 18 19:10:34.554878 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:34.554815    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:34.713071 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:34.713038    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:34.943420 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:34.943324    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:35.426837 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:35.426782    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:36.455706 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:36.455672    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:37.232143 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:37.232062    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:37.930401 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:37.930365    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:45.805085 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:45.805051    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:45.848222 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:45.848188    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz"]
Sep 18 19:10:45.848467 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:45.848424    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz" podUID="57e8a51c-44d7-43f8-ae87-2e61e74ef886" containerName="payload-pre-processing" containerID="cri-o://098c80549be205fc4bdf40964a5ab775eb2bf57d246e8c67a0e006fd608fb4a8" gracePeriod=30
Sep 18 19:10:45.878393 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:45.878361    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:45.954889 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:45.954843    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:46.028324 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.028198    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:46.096042 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.096016    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:46.097524 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.097505    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz"
Sep 18 19:10:46.198620 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.198586    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5hvf\" (UniqueName: \"kubernetes.io/projected/57e8a51c-44d7-43f8-ae87-2e61e74ef886-kube-api-access-m5hvf\") pod \"57e8a51c-44d7-43f8-ae87-2e61e74ef886\" (UID: \"57e8a51c-44d7-43f8-ae87-2e61e74ef886\") "
Sep 18 19:10:46.198783 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.198742    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57e8a51c-44d7-43f8-ae87-2e61e74ef886-plugins-config-volume\") pod \"57e8a51c-44d7-43f8-ae87-2e61e74ef886\" (UID: \"57e8a51c-44d7-43f8-ae87-2e61e74ef886\") "
Sep 18 19:10:46.199115 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.199089    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57e8a51c-44d7-43f8-ae87-2e61e74ef886-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "57e8a51c-44d7-43f8-ae87-2e61e74ef886" (UID: "57e8a51c-44d7-43f8-ae87-2e61e74ef886"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:10:46.200740 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.200715    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57e8a51c-44d7-43f8-ae87-2e61e74ef886-kube-api-access-m5hvf" (OuterVolumeSpecName: "kube-api-access-m5hvf") pod "57e8a51c-44d7-43f8-ae87-2e61e74ef886" (UID: "57e8a51c-44d7-43f8-ae87-2e61e74ef886"). InnerVolumeSpecName "kube-api-access-m5hvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:10:46.300229 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.300192    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57e8a51c-44d7-43f8-ae87-2e61e74ef886-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:10:46.300229 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.300221    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5hvf\" (UniqueName: \"kubernetes.io/projected/57e8a51c-44d7-43f8-ae87-2e61e74ef886-kube-api-access-m5hvf\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:10:46.412345 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.412230    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:46.511150 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.511119    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:46.647660 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.647623    2558 generic.go:356] "Generic (PLEG): container finished" podID="57e8a51c-44d7-43f8-ae87-2e61e74ef886" containerID="098c80549be205fc4bdf40964a5ab775eb2bf57d246e8c67a0e006fd608fb4a8" exitCode=0
Sep 18 19:10:46.647855 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.647663    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz" event={"ID":"57e8a51c-44d7-43f8-ae87-2e61e74ef886","Type":"ContainerDied","Data":"098c80549be205fc4bdf40964a5ab775eb2bf57d246e8c67a0e006fd608fb4a8"}
Sep 18 19:10:46.647855 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.647741    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz" event={"ID":"57e8a51c-44d7-43f8-ae87-2e61e74ef886","Type":"ContainerDied","Data":"8566727bdfd1c98161bcd3bf2486dddfee9980f9fc07f1fcde7ceac4870b67a3"}
Sep 18 19:10:46.647855 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.647673    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz"
Sep 18 19:10:46.647855 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.647764    2558 scope.go:117] "RemoveContainer" containerID="098c80549be205fc4bdf40964a5ab775eb2bf57d246e8c67a0e006fd608fb4a8"
Sep 18 19:10:46.657031 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.657016    2558 scope.go:117] "RemoveContainer" containerID="098c80549be205fc4bdf40964a5ab775eb2bf57d246e8c67a0e006fd608fb4a8"
Sep 18 19:10:46.657282 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:10:46.657266    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"098c80549be205fc4bdf40964a5ab775eb2bf57d246e8c67a0e006fd608fb4a8\": container with ID starting with 098c80549be205fc4bdf40964a5ab775eb2bf57d246e8c67a0e006fd608fb4a8 not found: ID does not exist" containerID="098c80549be205fc4bdf40964a5ab775eb2bf57d246e8c67a0e006fd608fb4a8"
Sep 18 19:10:46.657326 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.657290    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"098c80549be205fc4bdf40964a5ab775eb2bf57d246e8c67a0e006fd608fb4a8"} err="failed to get container status \"098c80549be205fc4bdf40964a5ab775eb2bf57d246e8c67a0e006fd608fb4a8\": rpc error: code = NotFound desc = could not find container \"098c80549be205fc4bdf40964a5ab775eb2bf57d246e8c67a0e006fd608fb4a8\": container with ID starting with 098c80549be205fc4bdf40964a5ab775eb2bf57d246e8c67a0e006fd608fb4a8 not found: ID does not exist"
Sep 18 19:10:46.673439 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.673392    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz"]
Sep 18 19:10:46.676923 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:46.676902    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-298125-5b87d487c4-fkjxz"]
Sep 18 19:10:47.170237 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:47.170204    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:47.844205 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:47.844168    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57e8a51c-44d7-43f8-ae87-2e61e74ef886" path="/var/lib/kubelet/pods/57e8a51c-44d7-43f8-ae87-2e61e74ef886/volumes"
Sep 18 19:10:52.358688 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:52.358645    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:52.477762 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:52.477728    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:52.654200 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:52.654113    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:53.353505 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:53.353473    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:54.162256 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:54.162221    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:10:54.713634 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:10:54.713600    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:11:03.529523 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:03.529489    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:11:03.562492 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:03.562460    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld"]
Sep 18 19:11:03.562718 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:03.562693    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld" podUID="358e8e15-d5aa-45e0-929c-05f7f90c2b0a" containerName="payload-pre-processing" containerID="cri-o://6864a5b0fe8e3a5261ff59032f19cd6d1abe752bd732c022a824c29123e665bc" gracePeriod=30
Sep 18 19:11:03.629269 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:03.629232    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:11:03.731081 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:03.731027    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:11:03.739176 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:03.739145    2558 generic.go:356] "Generic (PLEG): container finished" podID="358e8e15-d5aa-45e0-929c-05f7f90c2b0a" containerID="6864a5b0fe8e3a5261ff59032f19cd6d1abe752bd732c022a824c29123e665bc" exitCode=0
Sep 18 19:11:03.739315 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:03.739214    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld" event={"ID":"358e8e15-d5aa-45e0-929c-05f7f90c2b0a","Type":"ContainerDied","Data":"6864a5b0fe8e3a5261ff59032f19cd6d1abe752bd732c022a824c29123e665bc"}
Sep 18 19:11:03.825743 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:03.825714    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld"
Sep 18 19:11:03.831081 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:03.831028    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:11:03.962291 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:03.962257    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/358e8e15-d5aa-45e0-929c-05f7f90c2b0a-plugins-config-volume\") pod \"358e8e15-d5aa-45e0-929c-05f7f90c2b0a\" (UID: \"358e8e15-d5aa-45e0-929c-05f7f90c2b0a\") "
Sep 18 19:11:03.962465 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:03.962315    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgnbd\" (UniqueName: \"kubernetes.io/projected/358e8e15-d5aa-45e0-929c-05f7f90c2b0a-kube-api-access-wgnbd\") pod \"358e8e15-d5aa-45e0-929c-05f7f90c2b0a\" (UID: \"358e8e15-d5aa-45e0-929c-05f7f90c2b0a\") "
Sep 18 19:11:03.962703 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:03.962679    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/358e8e15-d5aa-45e0-929c-05f7f90c2b0a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "358e8e15-d5aa-45e0-929c-05f7f90c2b0a" (UID: "358e8e15-d5aa-45e0-929c-05f7f90c2b0a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:11:03.964579 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:03.964549    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/358e8e15-d5aa-45e0-929c-05f7f90c2b0a-kube-api-access-wgnbd" (OuterVolumeSpecName: "kube-api-access-wgnbd") pod "358e8e15-d5aa-45e0-929c-05f7f90c2b0a" (UID: "358e8e15-d5aa-45e0-929c-05f7f90c2b0a"). InnerVolumeSpecName "kube-api-access-wgnbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:11:04.063358 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:04.063254    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/358e8e15-d5aa-45e0-929c-05f7f90c2b0a-plugins-config-volume\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:11:04.063358 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:04.063288    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgnbd\" (UniqueName: \"kubernetes.io/projected/358e8e15-d5aa-45e0-929c-05f7f90c2b0a-kube-api-access-wgnbd\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:11:04.157627 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:04.157595    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:11:04.256627 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:04.256587    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:11:04.745555 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:04.745529    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld"
Sep 18 19:11:04.746014 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:04.745564    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld" event={"ID":"358e8e15-d5aa-45e0-929c-05f7f90c2b0a","Type":"ContainerDied","Data":"b7bb445763f2648b396161878135c5dcba9593b5a6d8b5df0ed3428224ca6fbb"}
Sep 18 19:11:04.746014 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:04.745623    2558 scope.go:117] "RemoveContainer" containerID="6864a5b0fe8e3a5261ff59032f19cd6d1abe752bd732c022a824c29123e665bc"
Sep 18 19:11:04.773120 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:04.773093    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld"]
Sep 18 19:11:04.779410 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:04.779384    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a6d509-6b54475d66-2fwld"]
Sep 18 19:11:05.843975 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:05.843942    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="358e8e15-d5aa-45e0-929c-05f7f90c2b0a" path="/var/lib/kubelet/pods/358e8e15-d5aa-45e0-929c-05f7f90c2b0a/volumes"
Sep 18 19:11:10.205895 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:10.205852    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:11:10.305945 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:10.305910    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:11:10.468786 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:10.468703    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:11:12.751637 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:12.751603    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64fbd675b6-l9gft"]
Sep 18 19:11:12.752127 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:12.751807    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-64fbd675b6-l9gft" podUID="939e465d-57fd-4592-a8e8-13abfabd5c1a" containerName="manager" containerID="cri-o://269dbb2548e2f7ca18e30ff775d1ee188ae896b7a347bfe7fc8ff04c897c7dd8" gracePeriod=30
Sep 18 19:11:12.994802 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:12.994775    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fbd675b6-l9gft"
Sep 18 19:11:13.144911 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.144814    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdjmh\" (UniqueName: \"kubernetes.io/projected/939e465d-57fd-4592-a8e8-13abfabd5c1a-kube-api-access-cdjmh\") pod \"939e465d-57fd-4592-a8e8-13abfabd5c1a\" (UID: \"939e465d-57fd-4592-a8e8-13abfabd5c1a\") "
Sep 18 19:11:13.145051 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.144922    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/939e465d-57fd-4592-a8e8-13abfabd5c1a-webhook-cert\") pod \"939e465d-57fd-4592-a8e8-13abfabd5c1a\" (UID: \"939e465d-57fd-4592-a8e8-13abfabd5c1a\") "
Sep 18 19:11:13.145051 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.145037    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/939e465d-57fd-4592-a8e8-13abfabd5c1a-metrics-certs\") pod \"939e465d-57fd-4592-a8e8-13abfabd5c1a\" (UID: \"939e465d-57fd-4592-a8e8-13abfabd5c1a\") "
Sep 18 19:11:13.146932 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.146902    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/939e465d-57fd-4592-a8e8-13abfabd5c1a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "939e465d-57fd-4592-a8e8-13abfabd5c1a" (UID: "939e465d-57fd-4592-a8e8-13abfabd5c1a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:11:13.146932 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.146905    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/939e465d-57fd-4592-a8e8-13abfabd5c1a-kube-api-access-cdjmh" (OuterVolumeSpecName: "kube-api-access-cdjmh") pod "939e465d-57fd-4592-a8e8-13abfabd5c1a" (UID: "939e465d-57fd-4592-a8e8-13abfabd5c1a"). InnerVolumeSpecName "kube-api-access-cdjmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:11:13.147101 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.146973    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/939e465d-57fd-4592-a8e8-13abfabd5c1a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "939e465d-57fd-4592-a8e8-13abfabd5c1a" (UID: "939e465d-57fd-4592-a8e8-13abfabd5c1a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:11:13.246149 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.246122    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdjmh\" (UniqueName: \"kubernetes.io/projected/939e465d-57fd-4592-a8e8-13abfabd5c1a-kube-api-access-cdjmh\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:11:13.246149 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.246146    2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/939e465d-57fd-4592-a8e8-13abfabd5c1a-webhook-cert\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:11:13.246149 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.246155    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/939e465d-57fd-4592-a8e8-13abfabd5c1a-metrics-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:11:13.792012 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.791976    2558 generic.go:356] "Generic (PLEG): container finished" podID="939e465d-57fd-4592-a8e8-13abfabd5c1a" containerID="269dbb2548e2f7ca18e30ff775d1ee188ae896b7a347bfe7fc8ff04c897c7dd8" exitCode=0
Sep 18 19:11:13.792475 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.792032    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fbd675b6-l9gft"
Sep 18 19:11:13.792475 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.792055    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fbd675b6-l9gft" event={"ID":"939e465d-57fd-4592-a8e8-13abfabd5c1a","Type":"ContainerDied","Data":"269dbb2548e2f7ca18e30ff775d1ee188ae896b7a347bfe7fc8ff04c897c7dd8"}
Sep 18 19:11:13.792475 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.792103    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fbd675b6-l9gft" event={"ID":"939e465d-57fd-4592-a8e8-13abfabd5c1a","Type":"ContainerDied","Data":"fed79a382ecf2b34c1aed2f73e434807a1e55c0f73cfa1d7ab5658b1731809a0"}
Sep 18 19:11:13.792475 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.792124    2558 scope.go:117] "RemoveContainer" containerID="269dbb2548e2f7ca18e30ff775d1ee188ae896b7a347bfe7fc8ff04c897c7dd8"
Sep 18 19:11:13.801620 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.801604    2558 scope.go:117] "RemoveContainer" containerID="269dbb2548e2f7ca18e30ff775d1ee188ae896b7a347bfe7fc8ff04c897c7dd8"
Sep 18 19:11:13.801897 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:11:13.801879    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"269dbb2548e2f7ca18e30ff775d1ee188ae896b7a347bfe7fc8ff04c897c7dd8\": container with ID starting with 269dbb2548e2f7ca18e30ff775d1ee188ae896b7a347bfe7fc8ff04c897c7dd8 not found: ID does not exist" containerID="269dbb2548e2f7ca18e30ff775d1ee188ae896b7a347bfe7fc8ff04c897c7dd8"
Sep 18 19:11:13.801959 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.801912    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"269dbb2548e2f7ca18e30ff775d1ee188ae896b7a347bfe7fc8ff04c897c7dd8"} err="failed to get container status \"269dbb2548e2f7ca18e30ff775d1ee188ae896b7a347bfe7fc8ff04c897c7dd8\": rpc error: code = NotFound desc = could not find container \"269dbb2548e2f7ca18e30ff775d1ee188ae896b7a347bfe7fc8ff04c897c7dd8\": container with ID starting with 269dbb2548e2f7ca18e30ff775d1ee188ae896b7a347bfe7fc8ff04c897c7dd8 not found: ID does not exist"
Sep 18 19:11:13.813906 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.813880    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64fbd675b6-l9gft"]
Sep 18 19:11:13.817351 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.817331    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-64fbd675b6-l9gft"]
Sep 18 19:11:13.844211 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:13.844189    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="939e465d-57fd-4592-a8e8-13abfabd5c1a" path="/var/lib/kubelet/pods/939e465d-57fd-4592-a8e8-13abfabd5c1a/volumes"
Sep 18 19:11:26.556458 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:11:26.556416    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:12:14.598980 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.598768    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64fbd675b6-tkkh9"]
Sep 18 19:12:14.600371 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.600343    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4732967a-67fc-49a0-87f3-8e602993ec54" containerName="maas-api"
Sep 18 19:12:14.600528 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.600517    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4732967a-67fc-49a0-87f3-8e602993ec54" containerName="maas-api"
Sep 18 19:12:14.600618 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.600608    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0596fbfe-bf4a-4adc-ae55-70a47b2c748a" containerName="payload-pre-processing"
Sep 18 19:12:14.600691 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.600683    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0596fbfe-bf4a-4adc-ae55-70a47b2c748a" containerName="payload-pre-processing"
Sep 18 19:12:14.600787 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.600778    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="358e8e15-d5aa-45e0-929c-05f7f90c2b0a" containerName="payload-pre-processing"
Sep 18 19:12:14.600885 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.600874    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="358e8e15-d5aa-45e0-929c-05f7f90c2b0a" containerName="payload-pre-processing"
Sep 18 19:12:14.600963 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.600954    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2d5c077-92fb-4206-8360-4e6492eb45ae" containerName="maas-api"
Sep 18 19:12:14.601036 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.601028    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2d5c077-92fb-4206-8360-4e6492eb45ae" containerName="maas-api"
Sep 18 19:12:14.601128 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.601119    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b00d12fb-4364-47a0-89eb-2f68505abdc8" containerName="payload-processing"
Sep 18 19:12:14.601197 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.601189    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b00d12fb-4364-47a0-89eb-2f68505abdc8" containerName="payload-processing"
Sep 18 19:12:14.601263 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.601255    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="939e465d-57fd-4592-a8e8-13abfabd5c1a" containerName="manager"
Sep 18 19:12:14.601344 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.601334    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="939e465d-57fd-4592-a8e8-13abfabd5c1a" containerName="manager"
Sep 18 19:12:14.601424 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.601412    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57e8a51c-44d7-43f8-ae87-2e61e74ef886" containerName="payload-pre-processing"
Sep 18 19:12:14.601490 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.601482    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="57e8a51c-44d7-43f8-ae87-2e61e74ef886" containerName="payload-pre-processing"
Sep 18 19:12:14.601554 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.601546    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="095736cf-0478-48c9-b0f1-bba353a00a42" containerName="payload-processing"
Sep 18 19:12:14.601617 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.601609    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="095736cf-0478-48c9-b0f1-bba353a00a42" containerName="payload-processing"
Sep 18 19:12:14.601709 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.601699    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d1261f7-766c-42fe-b061-98b4893c03d0" containerName="payload-pre-processing"
Sep 18 19:12:14.601784 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.601776    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d1261f7-766c-42fe-b061-98b4893c03d0" containerName="payload-pre-processing"
Sep 18 19:12:14.601900 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.601880    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79bfc37c-f7d4-4045-aec3-8bf8031901dd" containerName="payload-pre-processing"
Sep 18 19:12:14.601900 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.601896    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="79bfc37c-f7d4-4045-aec3-8bf8031901dd" containerName="payload-pre-processing"
Sep 18 19:12:14.602193 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.602167    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="939e465d-57fd-4592-a8e8-13abfabd5c1a" containerName="manager"
Sep 18 19:12:14.602260 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.602198    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="57e8a51c-44d7-43f8-ae87-2e61e74ef886" containerName="payload-pre-processing"
Sep 18 19:12:14.602260 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.602209    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4732967a-67fc-49a0-87f3-8e602993ec54" containerName="maas-api"
Sep 18 19:12:14.602260 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.602219    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0596fbfe-bf4a-4adc-ae55-70a47b2c748a" containerName="payload-pre-processing"
Sep 18 19:12:14.602260 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.602229    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="358e8e15-d5aa-45e0-929c-05f7f90c2b0a" containerName="payload-pre-processing"
Sep 18 19:12:14.602260 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.602239    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="79bfc37c-f7d4-4045-aec3-8bf8031901dd" containerName="payload-pre-processing"
Sep 18 19:12:14.602260 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.602250    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2d5c077-92fb-4206-8360-4e6492eb45ae" containerName="maas-api"
Sep 18 19:12:14.602260 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.602257    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="095736cf-0478-48c9-b0f1-bba353a00a42" containerName="payload-processing"
Sep 18 19:12:14.602557 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.602268    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b00d12fb-4364-47a0-89eb-2f68505abdc8" containerName="payload-processing"
Sep 18 19:12:14.602557 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.602277    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d1261f7-766c-42fe-b061-98b4893c03d0" containerName="payload-pre-processing"
Sep 18 19:12:14.605627 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.605608    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fbd675b6-tkkh9"
Sep 18 19:12:14.609246 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.609220    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-x7kbw\""
Sep 18 19:12:14.609340 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.609244    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 18 19:12:14.609428 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.609367    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 18 19:12:14.609488 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.609467    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 18 19:12:14.612144 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.612125    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64fbd675b6-tkkh9"]
Sep 18 19:12:14.695761 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.695735    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8sdb\" (UniqueName: \"kubernetes.io/projected/7e5b4621-6345-4706-92ef-70544a49d221-kube-api-access-w8sdb\") pod \"maas-controller-64fbd675b6-tkkh9\" (UID: \"7e5b4621-6345-4706-92ef-70544a49d221\") " pod="opendatahub/maas-controller-64fbd675b6-tkkh9"
Sep 18 19:12:14.695900 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.695770    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e5b4621-6345-4706-92ef-70544a49d221-webhook-cert\") pod \"maas-controller-64fbd675b6-tkkh9\" (UID: \"7e5b4621-6345-4706-92ef-70544a49d221\") " pod="opendatahub/maas-controller-64fbd675b6-tkkh9"
Sep 18 19:12:14.695900 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.695811    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e5b4621-6345-4706-92ef-70544a49d221-metrics-certs\") pod \"maas-controller-64fbd675b6-tkkh9\" (UID: \"7e5b4621-6345-4706-92ef-70544a49d221\") " pod="opendatahub/maas-controller-64fbd675b6-tkkh9"
Sep 18 19:12:14.796623 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.796592    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e5b4621-6345-4706-92ef-70544a49d221-metrics-certs\") pod \"maas-controller-64fbd675b6-tkkh9\" (UID: \"7e5b4621-6345-4706-92ef-70544a49d221\") " pod="opendatahub/maas-controller-64fbd675b6-tkkh9"
Sep 18 19:12:14.796766 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.796685    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8sdb\" (UniqueName: \"kubernetes.io/projected/7e5b4621-6345-4706-92ef-70544a49d221-kube-api-access-w8sdb\") pod \"maas-controller-64fbd675b6-tkkh9\" (UID: \"7e5b4621-6345-4706-92ef-70544a49d221\") " pod="opendatahub/maas-controller-64fbd675b6-tkkh9"
Sep 18 19:12:14.796766 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.796726    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e5b4621-6345-4706-92ef-70544a49d221-webhook-cert\") pod \"maas-controller-64fbd675b6-tkkh9\" (UID: \"7e5b4621-6345-4706-92ef-70544a49d221\") " pod="opendatahub/maas-controller-64fbd675b6-tkkh9"
Sep 18 19:12:14.799015 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.798996    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e5b4621-6345-4706-92ef-70544a49d221-webhook-cert\") pod \"maas-controller-64fbd675b6-tkkh9\" (UID: \"7e5b4621-6345-4706-92ef-70544a49d221\") " pod="opendatahub/maas-controller-64fbd675b6-tkkh9"
Sep 18 19:12:14.799119 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.799035    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e5b4621-6345-4706-92ef-70544a49d221-metrics-certs\") pod \"maas-controller-64fbd675b6-tkkh9\" (UID: \"7e5b4621-6345-4706-92ef-70544a49d221\") " pod="opendatahub/maas-controller-64fbd675b6-tkkh9"
Sep 18 19:12:14.805069 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.805050    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8sdb\" (UniqueName: \"kubernetes.io/projected/7e5b4621-6345-4706-92ef-70544a49d221-kube-api-access-w8sdb\") pod \"maas-controller-64fbd675b6-tkkh9\" (UID: \"7e5b4621-6345-4706-92ef-70544a49d221\") " pod="opendatahub/maas-controller-64fbd675b6-tkkh9"
Sep 18 19:12:14.917582 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:14.917516    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fbd675b6-tkkh9"
Sep 18 19:12:15.189788 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:15.189761    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64fbd675b6-tkkh9"]
Sep 18 19:12:15.191989 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:12:15.191959    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e5b4621_6345_4706_92ef_70544a49d221.slice/crio-7b80749ca88e205e83f2248032d657fcf0a528ba54c08eb324859ed3e55f7afa WatchSource:0}: Error finding container 7b80749ca88e205e83f2248032d657fcf0a528ba54c08eb324859ed3e55f7afa: Status 404 returned error can't find the container with id 7b80749ca88e205e83f2248032d657fcf0a528ba54c08eb324859ed3e55f7afa
Sep 18 19:12:15.193252 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:15.193234    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 19:12:16.085334 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:16.085294    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fbd675b6-tkkh9" event={"ID":"7e5b4621-6345-4706-92ef-70544a49d221","Type":"ContainerStarted","Data":"d8487cf787e4392b8b0dd0e805c13581387c7066ffb01fcfde777d3a4cbc32ea"}
Sep 18 19:12:16.085794 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:16.085346    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fbd675b6-tkkh9" event={"ID":"7e5b4621-6345-4706-92ef-70544a49d221","Type":"ContainerStarted","Data":"7b80749ca88e205e83f2248032d657fcf0a528ba54c08eb324859ed3e55f7afa"}
Sep 18 19:12:16.085794 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:16.085389    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64fbd675b6-tkkh9"
Sep 18 19:12:16.110394 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:16.110346    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64fbd675b6-tkkh9" podStartSLOduration=1.644046655 podStartE2EDuration="2.110332148s" totalImagesPullingTime="466.285493ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:12:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:12:16.108420909 +0000 UTC m=+1972.306905159" watchObservedRunningTime="2026-09-18 19:12:16.110332148 +0000 UTC m=+1972.308816351"
Sep 18 19:12:27.092459 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:27.092425    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64fbd675b6-tkkh9"
Sep 18 19:12:36.317727 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:36.317690    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:12:36.411776 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:36.411743    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64fbd675b6-tkkh9"]
Sep 18 19:12:36.412022 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:36.411997    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-64fbd675b6-tkkh9" podUID="7e5b4621-6345-4706-92ef-70544a49d221" containerName="manager" containerID="cri-o://d8487cf787e4392b8b0dd0e805c13581387c7066ffb01fcfde777d3a4cbc32ea" gracePeriod=30
Sep 18 19:12:36.656972 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:36.656948    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fbd675b6-tkkh9"
Sep 18 19:12:36.798960 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:36.798922    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e5b4621-6345-4706-92ef-70544a49d221-metrics-certs\") pod \"7e5b4621-6345-4706-92ef-70544a49d221\" (UID: \"7e5b4621-6345-4706-92ef-70544a49d221\") "
Sep 18 19:12:36.799127 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:36.799111    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8sdb\" (UniqueName: \"kubernetes.io/projected/7e5b4621-6345-4706-92ef-70544a49d221-kube-api-access-w8sdb\") pod \"7e5b4621-6345-4706-92ef-70544a49d221\" (UID: \"7e5b4621-6345-4706-92ef-70544a49d221\") "
Sep 18 19:12:36.799204 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:36.799191    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e5b4621-6345-4706-92ef-70544a49d221-webhook-cert\") pod \"7e5b4621-6345-4706-92ef-70544a49d221\" (UID: \"7e5b4621-6345-4706-92ef-70544a49d221\") "
Sep 18 19:12:36.800970 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:36.800939    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e5b4621-6345-4706-92ef-70544a49d221-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7e5b4621-6345-4706-92ef-70544a49d221" (UID: "7e5b4621-6345-4706-92ef-70544a49d221"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:12:36.801197 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:36.801175    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e5b4621-6345-4706-92ef-70544a49d221-kube-api-access-w8sdb" (OuterVolumeSpecName: "kube-api-access-w8sdb") pod "7e5b4621-6345-4706-92ef-70544a49d221" (UID: "7e5b4621-6345-4706-92ef-70544a49d221"). InnerVolumeSpecName "kube-api-access-w8sdb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:12:36.801249 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:36.801199    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e5b4621-6345-4706-92ef-70544a49d221-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7e5b4621-6345-4706-92ef-70544a49d221" (UID: "7e5b4621-6345-4706-92ef-70544a49d221"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:12:36.900921 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:36.900850    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e5b4621-6345-4706-92ef-70544a49d221-metrics-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:12:36.900921 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:36.900873    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8sdb\" (UniqueName: \"kubernetes.io/projected/7e5b4621-6345-4706-92ef-70544a49d221-kube-api-access-w8sdb\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:12:36.900921 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:36.900883    2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e5b4621-6345-4706-92ef-70544a49d221-webhook-cert\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:12:37.192780 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.192691    2558 generic.go:356] "Generic (PLEG): container finished" podID="7e5b4621-6345-4706-92ef-70544a49d221" containerID="d8487cf787e4392b8b0dd0e805c13581387c7066ffb01fcfde777d3a4cbc32ea" exitCode=0
Sep 18 19:12:37.192780 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.192738    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fbd675b6-tkkh9"
Sep 18 19:12:37.193009 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.192774    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fbd675b6-tkkh9" event={"ID":"7e5b4621-6345-4706-92ef-70544a49d221","Type":"ContainerDied","Data":"d8487cf787e4392b8b0dd0e805c13581387c7066ffb01fcfde777d3a4cbc32ea"}
Sep 18 19:12:37.193009 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.192839    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fbd675b6-tkkh9" event={"ID":"7e5b4621-6345-4706-92ef-70544a49d221","Type":"ContainerDied","Data":"7b80749ca88e205e83f2248032d657fcf0a528ba54c08eb324859ed3e55f7afa"}
Sep 18 19:12:37.193009 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.192863    2558 scope.go:117] "RemoveContainer" containerID="d8487cf787e4392b8b0dd0e805c13581387c7066ffb01fcfde777d3a4cbc32ea"
Sep 18 19:12:37.202224 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.202210    2558 scope.go:117] "RemoveContainer" containerID="d8487cf787e4392b8b0dd0e805c13581387c7066ffb01fcfde777d3a4cbc32ea"
Sep 18 19:12:37.202482 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:12:37.202461    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8487cf787e4392b8b0dd0e805c13581387c7066ffb01fcfde777d3a4cbc32ea\": container with ID starting with d8487cf787e4392b8b0dd0e805c13581387c7066ffb01fcfde777d3a4cbc32ea not found: ID does not exist" containerID="d8487cf787e4392b8b0dd0e805c13581387c7066ffb01fcfde777d3a4cbc32ea"
Sep 18 19:12:37.202547 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.202496    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8487cf787e4392b8b0dd0e805c13581387c7066ffb01fcfde777d3a4cbc32ea"} err="failed to get container status \"d8487cf787e4392b8b0dd0e805c13581387c7066ffb01fcfde777d3a4cbc32ea\": rpc error: code = NotFound desc = could not find container \"d8487cf787e4392b8b0dd0e805c13581387c7066ffb01fcfde777d3a4cbc32ea\": container with ID starting with d8487cf787e4392b8b0dd0e805c13581387c7066ffb01fcfde777d3a4cbc32ea not found: ID does not exist"
Sep 18 19:12:37.214323 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.214299    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64fbd675b6-tkkh9"]
Sep 18 19:12:37.219810 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.219786    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-64fbd675b6-tkkh9"]
Sep 18 19:12:37.846793 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.846755    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e5b4621-6345-4706-92ef-70544a49d221" path="/var/lib/kubelet/pods/7e5b4621-6345-4706-92ef-70544a49d221/volumes"
Sep 18 19:12:37.880615 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.880584    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:12:37.989269 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.989233    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64fbd675b6-5f5tg"]
Sep 18 19:12:37.989685 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.989668    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e5b4621-6345-4706-92ef-70544a49d221" containerName="manager"
Sep 18 19:12:37.989767 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.989686    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e5b4621-6345-4706-92ef-70544a49d221" containerName="manager"
Sep 18 19:12:37.989853 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.989775    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e5b4621-6345-4706-92ef-70544a49d221" containerName="manager"
Sep 18 19:12:37.994137 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.994118    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fbd675b6-5f5tg"
Sep 18 19:12:37.997621 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.997590    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 18 19:12:37.997731 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.997629    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 18 19:12:37.997731 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.997630    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-x7kbw\""
Sep 18 19:12:37.997731 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:37.997630    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 18 19:12:38.004054 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:38.004033    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64fbd675b6-5f5tg"]
Sep 18 19:12:38.111185 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:38.111108    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22-metrics-certs\") pod \"maas-controller-64fbd675b6-5f5tg\" (UID: \"c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22\") " pod="opendatahub/maas-controller-64fbd675b6-5f5tg"
Sep 18 19:12:38.111185 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:38.111155    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm7lq\" (UniqueName: \"kubernetes.io/projected/c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22-kube-api-access-mm7lq\") pod \"maas-controller-64fbd675b6-5f5tg\" (UID: \"c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22\") " pod="opendatahub/maas-controller-64fbd675b6-5f5tg"
Sep 18 19:12:38.111185 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:38.111188    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22-webhook-cert\") pod \"maas-controller-64fbd675b6-5f5tg\" (UID: \"c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22\") " pod="opendatahub/maas-controller-64fbd675b6-5f5tg"
Sep 18 19:12:38.212114 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:38.212085    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22-metrics-certs\") pod \"maas-controller-64fbd675b6-5f5tg\" (UID: \"c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22\") " pod="opendatahub/maas-controller-64fbd675b6-5f5tg"
Sep 18 19:12:38.212266 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:38.212129    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm7lq\" (UniqueName: \"kubernetes.io/projected/c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22-kube-api-access-mm7lq\") pod \"maas-controller-64fbd675b6-5f5tg\" (UID: \"c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22\") " pod="opendatahub/maas-controller-64fbd675b6-5f5tg"
Sep 18 19:12:38.212266 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:38.212204    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22-webhook-cert\") pod \"maas-controller-64fbd675b6-5f5tg\" (UID: \"c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22\") " pod="opendatahub/maas-controller-64fbd675b6-5f5tg"
Sep 18 19:12:38.214463 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:38.214439    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22-webhook-cert\") pod \"maas-controller-64fbd675b6-5f5tg\" (UID: \"c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22\") " pod="opendatahub/maas-controller-64fbd675b6-5f5tg"
Sep 18 19:12:38.214565 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:38.214499    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22-metrics-certs\") pod \"maas-controller-64fbd675b6-5f5tg\" (UID: \"c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22\") " pod="opendatahub/maas-controller-64fbd675b6-5f5tg"
Sep 18 19:12:38.221223 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:38.221201    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm7lq\" (UniqueName: \"kubernetes.io/projected/c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22-kube-api-access-mm7lq\") pod \"maas-controller-64fbd675b6-5f5tg\" (UID: \"c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22\") " pod="opendatahub/maas-controller-64fbd675b6-5f5tg"
Sep 18 19:12:38.275641 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:38.275614    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:12:38.305044 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:38.305021    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fbd675b6-5f5tg"
Sep 18 19:12:38.577149 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:38.577075    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64fbd675b6-5f5tg"]
Sep 18 19:12:38.579719 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:12:38.579695    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2e5dd2b_6fb6_483a_9d7c_08c0d6ca4f22.slice/crio-9bc9d7c1d3967f61febcb27ccd12fbcf474085d1758928d7eb0bd4b16c30d9f9 WatchSource:0}: Error finding container 9bc9d7c1d3967f61febcb27ccd12fbcf474085d1758928d7eb0bd4b16c30d9f9: Status 404 returned error can't find the container with id 9bc9d7c1d3967f61febcb27ccd12fbcf474085d1758928d7eb0bd4b16c30d9f9
Sep 18 19:12:39.205327 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:39.205233    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fbd675b6-5f5tg" event={"ID":"c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22","Type":"ContainerStarted","Data":"57323f44bedb8f0acd3de6243d7fbcef98ddf62621b36c2df294bef1a1812e3a"}
Sep 18 19:12:39.205327 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:39.205286    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fbd675b6-5f5tg" event={"ID":"c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22","Type":"ContainerStarted","Data":"9bc9d7c1d3967f61febcb27ccd12fbcf474085d1758928d7eb0bd4b16c30d9f9"}
Sep 18 19:12:39.205750 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:39.205346    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64fbd675b6-5f5tg"
Sep 18 19:12:39.234712 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:39.234650    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64fbd675b6-5f5tg" podStartSLOduration=1.8602661980000001 podStartE2EDuration="2.234631465s" totalImagesPullingTime="374.365267ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:12:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:12:39.232482593 +0000 UTC m=+1995.430966816" watchObservedRunningTime="2026-09-18 19:12:39.234631465 +0000 UTC m=+1995.433115670"
Sep 18 19:12:50.212963 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:12:50.212931    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64fbd675b6-5f5tg"
Sep 18 19:13:56.289258 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:13:56.289165    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:14:25.950497 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:14:25.950472    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bwsg_636f0979-fe2e-49fd-86d6-31bd7961e736/ovn-acl-logging/0.log"
Sep 18 19:14:25.955057 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:14:25.955035    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bwsg_636f0979-fe2e-49fd-86d6-31bd7961e736/ovn-acl-logging/0.log"
Sep 18 19:15:14.983736 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:14.983552    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:15:20.482790 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:20.482749    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:15:31.356193 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.356158    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"]
Sep 18 19:15:31.359725 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.359708    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.362504 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.362480    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-e93e024f-kserve-self-signed-certs\""
Sep 18 19:15:31.369732 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.369710    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"]
Sep 18 19:15:31.385996 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.385968    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:15:31.403105 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.403079    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-home\") pod \"e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") " pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.403208 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.403122    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a33c9719-cc85-4f0a-922a-0c4ef9a56693-tls-certs\") pod \"e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") " pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.403266 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.403205    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-tmp-dir\") pod \"e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") " pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.403266 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.403248    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8b5p\" (UniqueName: \"kubernetes.io/projected/a33c9719-cc85-4f0a-922a-0c4ef9a56693-kube-api-access-v8b5p\") pod \"e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") " pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.403346 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.403283    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-model-cache\") pod \"e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") " pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.403346 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.403337    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-dshm\") pod \"e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") " pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.504387 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.504348    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-dshm\") pod \"e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") " pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.504581 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.504394    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-home\") pod \"e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") " pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.504581 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.504431    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a33c9719-cc85-4f0a-922a-0c4ef9a56693-tls-certs\") pod \"e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") " pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.504581 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.504492    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-tmp-dir\") pod \"e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") " pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.504581 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.504528    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8b5p\" (UniqueName: \"kubernetes.io/projected/a33c9719-cc85-4f0a-922a-0c4ef9a56693-kube-api-access-v8b5p\") pod \"e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") " pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.504581 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.504558    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-model-cache\") pod \"e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") " pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.504877 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.504848    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-home\") pod \"e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") " pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.504945 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.504872    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-tmp-dir\") pod \"e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") " pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.505005 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.504965    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-model-cache\") pod \"e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") " pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.506594 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.506570    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-dshm\") pod \"e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") " pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.506992 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.506971    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a33c9719-cc85-4f0a-922a-0c4ef9a56693-tls-certs\") pod \"e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") " pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.519372 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.519351    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8b5p\" (UniqueName: \"kubernetes.io/projected/a33c9719-cc85-4f0a-922a-0c4ef9a56693-kube-api-access-v8b5p\") pod \"e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") " pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.669309 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.669231    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:31.943244 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:31.943217    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"]
Sep 18 19:15:31.945407 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:15:31.945365    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda33c9719_cc85_4f0a_922a_0c4ef9a56693.slice/crio-d6aa97e5cfbe0f05d507e03308c48879d6c9a33fb9058d07042e00a4e83d8dba WatchSource:0}: Error finding container d6aa97e5cfbe0f05d507e03308c48879d6c9a33fb9058d07042e00a4e83d8dba: Status 404 returned error can't find the container with id d6aa97e5cfbe0f05d507e03308c48879d6c9a33fb9058d07042e00a4e83d8dba
Sep 18 19:15:32.070055 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:32.070019    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj" event={"ID":"a33c9719-cc85-4f0a-922a-0c4ef9a56693","Type":"ContainerStarted","Data":"abb98e230f6d2bc5d5ac4c8d9397bb60c0415347fb5b5487ef3c8af3985d2002"}
Sep 18 19:15:32.070186 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:32.070071    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj" event={"ID":"a33c9719-cc85-4f0a-922a-0c4ef9a56693","Type":"ContainerStarted","Data":"d6aa97e5cfbe0f05d507e03308c48879d6c9a33fb9058d07042e00a4e83d8dba"}
Sep 18 19:15:32.087439 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:32.087395    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj" podStartSLOduration=1.087379827 podStartE2EDuration="1.087379827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:15:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:15:32.086110066 +0000 UTC m=+2168.284594295" watchObservedRunningTime="2026-09-18 19:15:32.087379827 +0000 UTC m=+2168.285864030"
Sep 18 19:15:41.669909 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:41.669875    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:41.670413 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:41.670141    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:41.682533 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:41.682504    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:42.131044 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:42.131013    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:15:44.525362 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:44.525328    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:15:53.758920 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:53.758888    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:15:54.468348 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:54.468302    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"]
Sep 18 19:15:54.494793 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:15:54.494759    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:16:01.612869 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:01.612830    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:16:01.713657 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:01.713628    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:16:09.488763 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.488705    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj" podUID="a33c9719-cc85-4f0a-922a-0c4ef9a56693" containerName="main" containerID="cri-o://abb98e230f6d2bc5d5ac4c8d9397bb60c0415347fb5b5487ef3c8af3985d2002" gracePeriod=45
Sep 18 19:16:09.734692 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.734666    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:16:09.863173 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.863147    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-home\") pod \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") "
Sep 18 19:16:09.863323 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.863182    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-dshm\") pod \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") "
Sep 18 19:16:09.863323 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.863215    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8b5p\" (UniqueName: \"kubernetes.io/projected/a33c9719-cc85-4f0a-922a-0c4ef9a56693-kube-api-access-v8b5p\") pod \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") "
Sep 18 19:16:09.863323 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.863249    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-tmp-dir\") pod \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") "
Sep 18 19:16:09.863323 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.863274    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-model-cache\") pod \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") "
Sep 18 19:16:09.863323 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.863312    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a33c9719-cc85-4f0a-922a-0c4ef9a56693-tls-certs\") pod \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\" (UID: \"a33c9719-cc85-4f0a-922a-0c4ef9a56693\") "
Sep 18 19:16:09.863581 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.863443    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-home" (OuterVolumeSpecName: "home") pod "a33c9719-cc85-4f0a-922a-0c4ef9a56693" (UID: "a33c9719-cc85-4f0a-922a-0c4ef9a56693"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:16:09.863581 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.863526    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-model-cache" (OuterVolumeSpecName: "model-cache") pod "a33c9719-cc85-4f0a-922a-0c4ef9a56693" (UID: "a33c9719-cc85-4f0a-922a-0c4ef9a56693"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:16:09.863581 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.863539    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a33c9719-cc85-4f0a-922a-0c4ef9a56693" (UID: "a33c9719-cc85-4f0a-922a-0c4ef9a56693"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:16:09.863739 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.863714    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-home\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:16:09.863778 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.863754    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-tmp-dir\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:16:09.863778 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.863772    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-model-cache\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:16:09.865412 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.865374    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-dshm" (OuterVolumeSpecName: "dshm") pod "a33c9719-cc85-4f0a-922a-0c4ef9a56693" (UID: "a33c9719-cc85-4f0a-922a-0c4ef9a56693"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:16:09.865543 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.865519    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a33c9719-cc85-4f0a-922a-0c4ef9a56693-kube-api-access-v8b5p" (OuterVolumeSpecName: "kube-api-access-v8b5p") pod "a33c9719-cc85-4f0a-922a-0c4ef9a56693" (UID: "a33c9719-cc85-4f0a-922a-0c4ef9a56693"). InnerVolumeSpecName "kube-api-access-v8b5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:16:09.865543 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.865519    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a33c9719-cc85-4f0a-922a-0c4ef9a56693-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a33c9719-cc85-4f0a-922a-0c4ef9a56693" (UID: "a33c9719-cc85-4f0a-922a-0c4ef9a56693"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:16:09.964604 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.964573    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a33c9719-cc85-4f0a-922a-0c4ef9a56693-dshm\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:16:09.964604 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.964600    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8b5p\" (UniqueName: \"kubernetes.io/projected/a33c9719-cc85-4f0a-922a-0c4ef9a56693-kube-api-access-v8b5p\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:16:09.964859 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:09.964610    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a33c9719-cc85-4f0a-922a-0c4ef9a56693-tls-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:16:10.266504 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.266471    2558 generic.go:356] "Generic (PLEG): container finished" podID="a33c9719-cc85-4f0a-922a-0c4ef9a56693" containerID="abb98e230f6d2bc5d5ac4c8d9397bb60c0415347fb5b5487ef3c8af3985d2002" exitCode=0
Sep 18 19:16:10.266673 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.266507    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj" event={"ID":"a33c9719-cc85-4f0a-922a-0c4ef9a56693","Type":"ContainerDied","Data":"abb98e230f6d2bc5d5ac4c8d9397bb60c0415347fb5b5487ef3c8af3985d2002"}
Sep 18 19:16:10.266673 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.266520    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"
Sep 18 19:16:10.266673 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.266553    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj" event={"ID":"a33c9719-cc85-4f0a-922a-0c4ef9a56693","Type":"ContainerDied","Data":"d6aa97e5cfbe0f05d507e03308c48879d6c9a33fb9058d07042e00a4e83d8dba"}
Sep 18 19:16:10.266673 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.266584    2558 scope.go:117] "RemoveContainer" containerID="abb98e230f6d2bc5d5ac4c8d9397bb60c0415347fb5b5487ef3c8af3985d2002"
Sep 18 19:16:10.277046 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.277029    2558 scope.go:117] "RemoveContainer" containerID="abb98e230f6d2bc5d5ac4c8d9397bb60c0415347fb5b5487ef3c8af3985d2002"
Sep 18 19:16:10.277332 ip-10-0-130-118 kubenswrapper[2558]: E0918 19:16:10.277310    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abb98e230f6d2bc5d5ac4c8d9397bb60c0415347fb5b5487ef3c8af3985d2002\": container with ID starting with abb98e230f6d2bc5d5ac4c8d9397bb60c0415347fb5b5487ef3c8af3985d2002 not found: ID does not exist" containerID="abb98e230f6d2bc5d5ac4c8d9397bb60c0415347fb5b5487ef3c8af3985d2002"
Sep 18 19:16:10.277436 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.277337    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abb98e230f6d2bc5d5ac4c8d9397bb60c0415347fb5b5487ef3c8af3985d2002"} err="failed to get container status \"abb98e230f6d2bc5d5ac4c8d9397bb60c0415347fb5b5487ef3c8af3985d2002\": rpc error: code = NotFound desc = could not find container \"abb98e230f6d2bc5d5ac4c8d9397bb60c0415347fb5b5487ef3c8af3985d2002\": container with ID starting with abb98e230f6d2bc5d5ac4c8d9397bb60c0415347fb5b5487ef3c8af3985d2002 not found: ID does not exist"
Sep 18 19:16:10.285429 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.285409    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"]
Sep 18 19:16:10.285787 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.285776    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a33c9719-cc85-4f0a-922a-0c4ef9a56693" containerName="main"
Sep 18 19:16:10.285842 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.285789    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a33c9719-cc85-4f0a-922a-0c4ef9a56693" containerName="main"
Sep 18 19:16:10.285896 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.285885    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a33c9719-cc85-4f0a-922a-0c4ef9a56693" containerName="main"
Sep 18 19:16:10.290907 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.290885    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"
Sep 18 19:16:10.303842 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.303770    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"]
Sep 18 19:16:10.315869 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.315840    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"]
Sep 18 19:16:10.320021 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.319999    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-e93e024f-kserve-6cd948dc99-phnnj"]
Sep 18 19:16:10.468414 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.468377    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98bdb9ab-ab28-4e0f-9427-b68535a01da8-metrics-certs\") pod \"maas-api-86d9ddc875-kh4bf\" (UID: \"98bdb9ab-ab28-4e0f-9427-b68535a01da8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"
Sep 18 19:16:10.468414 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.468419    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98bdb9ab-ab28-4e0f-9427-b68535a01da8-maas-api-tls\") pod \"maas-api-86d9ddc875-kh4bf\" (UID: \"98bdb9ab-ab28-4e0f-9427-b68535a01da8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"
Sep 18 19:16:10.468637 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.468451    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l97cr\" (UniqueName: \"kubernetes.io/projected/98bdb9ab-ab28-4e0f-9427-b68535a01da8-kube-api-access-l97cr\") pod \"maas-api-86d9ddc875-kh4bf\" (UID: \"98bdb9ab-ab28-4e0f-9427-b68535a01da8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"
Sep 18 19:16:10.468637 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.468556    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98bdb9ab-ab28-4e0f-9427-b68535a01da8-openshift-service-ca\") pod \"maas-api-86d9ddc875-kh4bf\" (UID: \"98bdb9ab-ab28-4e0f-9427-b68535a01da8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"
Sep 18 19:16:10.569920 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.569808    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98bdb9ab-ab28-4e0f-9427-b68535a01da8-maas-api-tls\") pod \"maas-api-86d9ddc875-kh4bf\" (UID: \"98bdb9ab-ab28-4e0f-9427-b68535a01da8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"
Sep 18 19:16:10.569920 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.569900    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l97cr\" (UniqueName: \"kubernetes.io/projected/98bdb9ab-ab28-4e0f-9427-b68535a01da8-kube-api-access-l97cr\") pod \"maas-api-86d9ddc875-kh4bf\" (UID: \"98bdb9ab-ab28-4e0f-9427-b68535a01da8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"
Sep 18 19:16:10.570405 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.569978    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98bdb9ab-ab28-4e0f-9427-b68535a01da8-openshift-service-ca\") pod \"maas-api-86d9ddc875-kh4bf\" (UID: \"98bdb9ab-ab28-4e0f-9427-b68535a01da8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"
Sep 18 19:16:10.570405 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.570029    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98bdb9ab-ab28-4e0f-9427-b68535a01da8-metrics-certs\") pod \"maas-api-86d9ddc875-kh4bf\" (UID: \"98bdb9ab-ab28-4e0f-9427-b68535a01da8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"
Sep 18 19:16:10.570662 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.570642    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98bdb9ab-ab28-4e0f-9427-b68535a01da8-openshift-service-ca\") pod \"maas-api-86d9ddc875-kh4bf\" (UID: \"98bdb9ab-ab28-4e0f-9427-b68535a01da8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"
Sep 18 19:16:10.572295 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.572261    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98bdb9ab-ab28-4e0f-9427-b68535a01da8-maas-api-tls\") pod \"maas-api-86d9ddc875-kh4bf\" (UID: \"98bdb9ab-ab28-4e0f-9427-b68535a01da8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"
Sep 18 19:16:10.572421 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.572316    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98bdb9ab-ab28-4e0f-9427-b68535a01da8-metrics-certs\") pod \"maas-api-86d9ddc875-kh4bf\" (UID: \"98bdb9ab-ab28-4e0f-9427-b68535a01da8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"
Sep 18 19:16:10.581381 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.581354    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l97cr\" (UniqueName: \"kubernetes.io/projected/98bdb9ab-ab28-4e0f-9427-b68535a01da8-kube-api-access-l97cr\") pod \"maas-api-86d9ddc875-kh4bf\" (UID: \"98bdb9ab-ab28-4e0f-9427-b68535a01da8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"
Sep 18 19:16:10.605166 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.605145    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"
Sep 18 19:16:10.877957 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:10.877929    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"]
Sep 18 19:16:10.879899 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:16:10.879874    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98bdb9ab_ab28_4e0f_9427_b68535a01da8.slice/crio-df51a2b57935ef50b7b42091ce466d2061052413af3b60a4f25206c9a533be9d WatchSource:0}: Error finding container df51a2b57935ef50b7b42091ce466d2061052413af3b60a4f25206c9a533be9d: Status 404 returned error can't find the container with id df51a2b57935ef50b7b42091ce466d2061052413af3b60a4f25206c9a533be9d
Sep 18 19:16:11.283179 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:11.283130    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf" event={"ID":"98bdb9ab-ab28-4e0f-9427-b68535a01da8","Type":"ContainerStarted","Data":"0ef9665e168b2c7ac9aa7f3a8c02f71678eeecbd508bfe45a3c3833c5a9a8b58"}
Sep 18 19:16:11.283355 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:11.283225    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf" event={"ID":"98bdb9ab-ab28-4e0f-9427-b68535a01da8","Type":"ContainerStarted","Data":"df51a2b57935ef50b7b42091ce466d2061052413af3b60a4f25206c9a533be9d"}
Sep 18 19:16:11.283355 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:11.283255    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"
Sep 18 19:16:11.301571 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:11.301526    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf" podStartSLOduration=0.984979543 podStartE2EDuration="1.301510397s" totalImagesPullingTime="316.530854ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:16:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:16:11.300319972 +0000 UTC m=+2207.498804232" watchObservedRunningTime="2026-09-18 19:16:11.301510397 +0000 UTC m=+2207.499994601"
Sep 18 19:16:11.845762 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:11.845727    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a33c9719-cc85-4f0a-922a-0c4ef9a56693" path="/var/lib/kubelet/pods/a33c9719-cc85-4f0a-922a-0c4ef9a56693/volumes"
Sep 18 19:16:17.298075 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:17.298047    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-kh4bf"
Sep 18 19:16:17.994298 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:17.994259    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:16:18.095326 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:18.095288    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:16:24.752451 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:24.752418    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"]
Sep 18 19:16:24.752849 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:24.752611    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5" podUID="2d69ce8d-b45d-48d9-a741-03dd8d21a684" containerName="maas-api" containerID="cri-o://ba7a1c60c8a3b804ac744522b787ea85879c42810a48c92b77b8fb43c1b3cc60" gracePeriod=30
Sep 18 19:16:25.993231 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:25.993203    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 19:16:26.103472 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:26.103399    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d69ce8d-b45d-48d9-a741-03dd8d21a684-metrics-certs\") pod \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\" (UID: \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\") "
Sep 18 19:16:26.103472 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:26.103451    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d69ce8d-b45d-48d9-a741-03dd8d21a684-openshift-service-ca\") pod \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\" (UID: \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\") "
Sep 18 19:16:26.103472 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:26.103474    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d69ce8d-b45d-48d9-a741-03dd8d21a684-maas-api-tls\") pod \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\" (UID: \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\") "
Sep 18 19:16:26.103754 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:26.103495    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n76mp\" (UniqueName: \"kubernetes.io/projected/2d69ce8d-b45d-48d9-a741-03dd8d21a684-kube-api-access-n76mp\") pod \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\" (UID: \"2d69ce8d-b45d-48d9-a741-03dd8d21a684\") "
Sep 18 19:16:26.103979 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:26.103954    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d69ce8d-b45d-48d9-a741-03dd8d21a684-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2d69ce8d-b45d-48d9-a741-03dd8d21a684" (UID: "2d69ce8d-b45d-48d9-a741-03dd8d21a684"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:16:26.105515 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:26.105490    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d69ce8d-b45d-48d9-a741-03dd8d21a684-kube-api-access-n76mp" (OuterVolumeSpecName: "kube-api-access-n76mp") pod "2d69ce8d-b45d-48d9-a741-03dd8d21a684" (UID: "2d69ce8d-b45d-48d9-a741-03dd8d21a684"). InnerVolumeSpecName "kube-api-access-n76mp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:16:26.105515 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:26.105506    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d69ce8d-b45d-48d9-a741-03dd8d21a684-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2d69ce8d-b45d-48d9-a741-03dd8d21a684" (UID: "2d69ce8d-b45d-48d9-a741-03dd8d21a684"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:16:26.105753 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:26.105535    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d69ce8d-b45d-48d9-a741-03dd8d21a684-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2d69ce8d-b45d-48d9-a741-03dd8d21a684" (UID: "2d69ce8d-b45d-48d9-a741-03dd8d21a684"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:16:26.201168 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:26.201140    2558 scope.go:117] "RemoveContainer" containerID="ba7a1c60c8a3b804ac744522b787ea85879c42810a48c92b77b8fb43c1b3cc60"
Sep 18 19:16:26.204608 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:26.204588    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d69ce8d-b45d-48d9-a741-03dd8d21a684-maas-api-tls\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:16:26.204680 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:26.204611    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n76mp\" (UniqueName: \"kubernetes.io/projected/2d69ce8d-b45d-48d9-a741-03dd8d21a684-kube-api-access-n76mp\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:16:26.204680 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:26.204621    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d69ce8d-b45d-48d9-a741-03dd8d21a684-metrics-certs\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:16:26.204680 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:26.204629    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d69ce8d-b45d-48d9-a741-03dd8d21a684-openshift-service-ca\") on node \"ip-10-0-130-118.ec2.internal\" DevicePath \"\""
Sep 18 19:16:26.361734 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:26.361644    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"
Sep 18 19:16:26.361942 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:26.361636    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5" event={"ID":"2d69ce8d-b45d-48d9-a741-03dd8d21a684","Type":"ContainerDied","Data":"ba7a1c60c8a3b804ac744522b787ea85879c42810a48c92b77b8fb43c1b3cc60"}
Sep 18 19:16:26.361942 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:26.361773    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5" event={"ID":"2d69ce8d-b45d-48d9-a741-03dd8d21a684","Type":"ContainerDied","Data":"4992013b8423bd1a1fed1b73c943a3ed5e6fd9592078ed1d63a6958a50d3ecc3"}
Sep 18 19:16:26.397459 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:26.397429    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"]
Sep 18 19:16:26.401208 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:26.401184    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-zx4j5"]
Sep 18 19:16:27.845160 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:27.845124    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d69ce8d-b45d-48d9-a741-03dd8d21a684" path="/var/lib/kubelet/pods/2d69ce8d-b45d-48d9-a741-03dd8d21a684/volumes"
Sep 18 19:16:41.605914 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:41.605803    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:16:41.684734 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:41.684697    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:16:44.622237 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:44.622198    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2h6z8_6b192a2e-3607-45c8-97aa-349c8b9503e0/global-pull-secret-syncer/0.log"
Sep 18 19:16:44.799003 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:44.798978    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log"
Sep 18 19:16:44.906260 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:44.906181    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-118.ec2.internal_396528f9074091ccc13f49651d4bd352/haproxy/0.log"
Sep 18 19:16:49.111440 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:49.111406    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:16:49.131920 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:49.131891    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d4dc55d7c-wdjw8_50d75267-8a6d-4bd5-9070-1df3879af1ab/authorino/0.log"
Sep 18 19:16:49.183120 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:49.183087    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fj9b_3bd977ef-b9c9-4380-923a-57758db726b1/konnectivity-agent/0.log" limit=1024
Sep 18 19:16:49.219008 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:49.218988    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-87dhq_d2086ce3-6af6-4353-b9ff-2004a8f4ee82/kuadrant-console-plugin/0.log"
Sep 18 19:16:51.066899 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:51.066870    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_567c4d29-eadb-49fc-ba4a-65f9cedabb99/alertmanager/0.log"
Sep 18 19:16:51.097924 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:51.097893    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_567c4d29-eadb-49fc-ba4a-65f9cedabb99/config-reloader/0.log"
Sep 18 19:16:51.142578 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:51.142548    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_567c4d29-eadb-49fc-ba4a-65f9cedabb99/kube-rbac-proxy-web/0.log"
Sep 18 19:16:51.169996 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:51.169977    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_567c4d29-eadb-49fc-ba4a-65f9cedabb99/kube-rbac-proxy/0.log"
Sep 18 19:16:51.199793 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:51.199773    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_567c4d29-eadb-49fc-ba4a-65f9cedabb99/kube-rbac-proxy-metric/0.log"
Sep 18 19:16:51.229173 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:51.229147    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_567c4d29-eadb-49fc-ba4a-65f9cedabb99/prom-label-proxy/0.log"
Sep 18 19:16:51.274384 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:51.274360    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_567c4d29-eadb-49fc-ba4a-65f9cedabb99/init-config-reloader/0.log"
Sep 18 19:16:51.382671 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:51.382598    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-57gn7_6d483a11-8755-4322-8843-d02a7f3d8679/kube-state-metrics/0.log"
Sep 18 19:16:51.421966 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:51.421937    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-57gn7_6d483a11-8755-4322-8843-d02a7f3d8679/kube-rbac-proxy-main/0.log"
Sep 18 19:16:51.468807 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:51.468776    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-57gn7_6d483a11-8755-4322-8843-d02a7f3d8679/kube-rbac-proxy-self/0.log"
Sep 18 19:16:51.540585 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:51.540551    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-cf66457bd-lgpm9_3ba18825-d735-4bac-9364-fc1fd3202f1f/metrics-server/0.log"
Sep 18 19:16:51.643191 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:51.643122    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5984459ff8-hp8n9_c93117d0-c124-4cf8-837f-d9631802de61/monitoring-plugin/0.log"
Sep 18 19:16:51.790236 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:51.790212    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6c75k_aefada7f-11b5-4e54-b38a-2cf8ddec87eb/node-exporter/0.log"
Sep 18 19:16:51.851416 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:51.851392    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6c75k_aefada7f-11b5-4e54-b38a-2cf8ddec87eb/kube-rbac-proxy/0.log"
Sep 18 19:16:51.896767 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:51.896696    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6c75k_aefada7f-11b5-4e54-b38a-2cf8ddec87eb/init-textfile/0.log"
Sep 18 19:16:52.215792 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:52.215701    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-dx96s_0e26bc53-6ab6-426b-ab64-2c369f52c9d7/kube-rbac-proxy-main/0.log"
Sep 18 19:16:52.262991 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:52.262965    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-dx96s_0e26bc53-6ab6-426b-ab64-2c369f52c9d7/kube-rbac-proxy-self/0.log"
Sep 18 19:16:52.335697 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:52.335672    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-dx96s_0e26bc53-6ab6-426b-ab64-2c369f52c9d7/openshift-state-metrics/0.log"
Sep 18 19:16:52.404976 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:52.404951    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0ed42c56-fa85-4e76-9381-c22373a94741/prometheus/0.log"
Sep 18 19:16:52.478100 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:52.478012    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0ed42c56-fa85-4e76-9381-c22373a94741/config-reloader/0.log"
Sep 18 19:16:52.536174 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:52.536130    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0ed42c56-fa85-4e76-9381-c22373a94741/thanos-sidecar/0.log"
Sep 18 19:16:52.596292 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:52.596263    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0ed42c56-fa85-4e76-9381-c22373a94741/kube-rbac-proxy-web/0.log"
Sep 18 19:16:52.633308 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:52.633275    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0ed42c56-fa85-4e76-9381-c22373a94741/kube-rbac-proxy/0.log"
Sep 18 19:16:52.658327 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:52.658306    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0ed42c56-fa85-4e76-9381-c22373a94741/kube-rbac-proxy-thanos/0.log"
Sep 18 19:16:52.693121 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:52.693093    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0ed42c56-fa85-4e76-9381-c22373a94741/init-config-reloader/0.log"
Sep 18 19:16:52.726100 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:52.726072    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-bjbwv_b891bf0e-ded5-4af2-aa7a-d59098eb82de/prometheus-operator/0.log"
Sep 18 19:16:52.745921 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:52.745900    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-bjbwv_b891bf0e-ded5-4af2-aa7a-d59098eb82de/kube-rbac-proxy/0.log"
Sep 18 19:16:52.771091 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:52.771047    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5c6747dc7b-g5kfj_47b9cc1a-d753-4187-b8db-f44aa9966171/prometheus-operator-admission-webhook/0.log"
Sep 18 19:16:52.885751 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:52.885705    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-549fdb4b7f-gvd8b_ea5ee20d-3bb1-4b88-9f09-12797e833989/thanos-query/0.log"
Sep 18 19:16:52.914279 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:52.914243    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-549fdb4b7f-gvd8b_ea5ee20d-3bb1-4b88-9f09-12797e833989/kube-rbac-proxy-web/0.log"
Sep 18 19:16:52.935843 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:52.935798    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-549fdb4b7f-gvd8b_ea5ee20d-3bb1-4b88-9f09-12797e833989/kube-rbac-proxy/0.log"
Sep 18 19:16:52.963714 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:52.963690    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-549fdb4b7f-gvd8b_ea5ee20d-3bb1-4b88-9f09-12797e833989/prom-label-proxy/0.log"
Sep 18 19:16:52.983454 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:52.983426    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-549fdb4b7f-gvd8b_ea5ee20d-3bb1-4b88-9f09-12797e833989/kube-rbac-proxy-rules/0.log"
Sep 18 19:16:53.017678 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:53.017611    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-549fdb4b7f-gvd8b_ea5ee20d-3bb1-4b88-9f09-12797e833989/kube-rbac-proxy-metrics/0.log"
Sep 18 19:16:54.503722 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.503687    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"]
Sep 18 19:16:54.504137 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.504122    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d69ce8d-b45d-48d9-a741-03dd8d21a684" containerName="maas-api"
Sep 18 19:16:54.504181 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.504138    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d69ce8d-b45d-48d9-a741-03dd8d21a684" containerName="maas-api"
Sep 18 19:16:54.504222 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.504202    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d69ce8d-b45d-48d9-a741-03dd8d21a684" containerName="maas-api"
Sep 18 19:16:54.507447 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.507430    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:16:54.510027 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.510010    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g6kgh\"/\"default-dockercfg-xkzgg\""
Sep 18 19:16:54.510203 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.510185    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g6kgh\"/\"openshift-service-ca.crt\""
Sep 18 19:16:54.511466 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.511452    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g6kgh\"/\"kube-root-ca.crt\""
Sep 18 19:16:54.533110 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.533089    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"]
Sep 18 19:16:54.564995 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.564971    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ae2fea9-ccfd-4357-808d-7515a3404aa1-lib-modules\") pod \"perf-node-gather-daemonset-jc96p\" (UID: \"5ae2fea9-ccfd-4357-808d-7515a3404aa1\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:16:54.565105 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.565015    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ae2fea9-ccfd-4357-808d-7515a3404aa1-proc\") pod \"perf-node-gather-daemonset-jc96p\" (UID: \"5ae2fea9-ccfd-4357-808d-7515a3404aa1\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:16:54.565105 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.565081    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfzqp\" (UniqueName: \"kubernetes.io/projected/5ae2fea9-ccfd-4357-808d-7515a3404aa1-kube-api-access-gfzqp\") pod \"perf-node-gather-daemonset-jc96p\" (UID: \"5ae2fea9-ccfd-4357-808d-7515a3404aa1\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:16:54.565180 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.565129    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ae2fea9-ccfd-4357-808d-7515a3404aa1-sys\") pod \"perf-node-gather-daemonset-jc96p\" (UID: \"5ae2fea9-ccfd-4357-808d-7515a3404aa1\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:16:54.565180 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.565159    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ae2fea9-ccfd-4357-808d-7515a3404aa1-podres\") pod \"perf-node-gather-daemonset-jc96p\" (UID: \"5ae2fea9-ccfd-4357-808d-7515a3404aa1\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:16:54.666125 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.666096    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfzqp\" (UniqueName: \"kubernetes.io/projected/5ae2fea9-ccfd-4357-808d-7515a3404aa1-kube-api-access-gfzqp\") pod \"perf-node-gather-daemonset-jc96p\" (UID: \"5ae2fea9-ccfd-4357-808d-7515a3404aa1\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:16:54.666276 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.666160    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ae2fea9-ccfd-4357-808d-7515a3404aa1-sys\") pod \"perf-node-gather-daemonset-jc96p\" (UID: \"5ae2fea9-ccfd-4357-808d-7515a3404aa1\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:16:54.666276 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.666200    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ae2fea9-ccfd-4357-808d-7515a3404aa1-podres\") pod \"perf-node-gather-daemonset-jc96p\" (UID: \"5ae2fea9-ccfd-4357-808d-7515a3404aa1\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:16:54.666276 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.666233    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ae2fea9-ccfd-4357-808d-7515a3404aa1-lib-modules\") pod \"perf-node-gather-daemonset-jc96p\" (UID: \"5ae2fea9-ccfd-4357-808d-7515a3404aa1\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:16:54.666276 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.666272    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ae2fea9-ccfd-4357-808d-7515a3404aa1-proc\") pod \"perf-node-gather-daemonset-jc96p\" (UID: \"5ae2fea9-ccfd-4357-808d-7515a3404aa1\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:16:54.666491 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.666292    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ae2fea9-ccfd-4357-808d-7515a3404aa1-sys\") pod \"perf-node-gather-daemonset-jc96p\" (UID: \"5ae2fea9-ccfd-4357-808d-7515a3404aa1\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:16:54.666491 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.666341    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ae2fea9-ccfd-4357-808d-7515a3404aa1-podres\") pod \"perf-node-gather-daemonset-jc96p\" (UID: \"5ae2fea9-ccfd-4357-808d-7515a3404aa1\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:16:54.666491 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.666384    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ae2fea9-ccfd-4357-808d-7515a3404aa1-proc\") pod \"perf-node-gather-daemonset-jc96p\" (UID: \"5ae2fea9-ccfd-4357-808d-7515a3404aa1\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:16:54.666491 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.666469    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ae2fea9-ccfd-4357-808d-7515a3404aa1-lib-modules\") pod \"perf-node-gather-daemonset-jc96p\" (UID: \"5ae2fea9-ccfd-4357-808d-7515a3404aa1\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:16:54.675145 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.675125    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfzqp\" (UniqueName: \"kubernetes.io/projected/5ae2fea9-ccfd-4357-808d-7515a3404aa1-kube-api-access-gfzqp\") pod \"perf-node-gather-daemonset-jc96p\" (UID: \"5ae2fea9-ccfd-4357-808d-7515a3404aa1\") " pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:16:54.817745 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:54.817676    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:16:55.111650 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:55.111575    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"]
Sep 18 19:16:55.115119 ip-10-0-130-118 kubenswrapper[2558]: W0918 19:16:55.115089    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5ae2fea9_ccfd_4357_808d_7515a3404aa1.slice/crio-df0f1452c4a75ff96c0764fa937bf964fb61b2ea42cb77e9d8ec28ff211f4a7f WatchSource:0}: Error finding container df0f1452c4a75ff96c0764fa937bf964fb61b2ea42cb77e9d8ec28ff211f4a7f: Status 404 returned error can't find the container with id df0f1452c4a75ff96c0764fa937bf964fb61b2ea42cb77e9d8ec28ff211f4a7f
Sep 18 19:16:55.504100 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:55.504067    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p" event={"ID":"5ae2fea9-ccfd-4357-808d-7515a3404aa1","Type":"ContainerStarted","Data":"d3bba674f4f56f64c657f314ccb45a19c20be067ac05e2587f8ca09b5813c6de"}
Sep 18 19:16:55.504426 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:55.504115    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p" event={"ID":"5ae2fea9-ccfd-4357-808d-7515a3404aa1","Type":"ContainerStarted","Data":"df0f1452c4a75ff96c0764fa937bf964fb61b2ea42cb77e9d8ec28ff211f4a7f"}
Sep 18 19:16:55.504426 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:55.504195    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:16:55.523682 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:55.523638    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p" podStartSLOduration=1.5236261899999999 podStartE2EDuration="1.52362619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:16:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:16:55.522435926 +0000 UTC m=+2251.720920151" watchObservedRunningTime="2026-09-18 19:16:55.52362619 +0000 UTC m=+2251.722110395"
Sep 18 19:16:55.675151 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:55.675123    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-569c544cc6-r5wff_52d908ad-92a4-4cdb-a279-c71290e038b3/console/0.log"
Sep 18 19:16:57.779920 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:57.779886    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7zhgc_b6973e0c-a5c1-400f-b6e1-734332e9ef35/dns/0.log"
Sep 18 19:16:57.817031 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:57.817008    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7zhgc_b6973e0c-a5c1-400f-b6e1-734332e9ef35/kube-rbac-proxy/0.log"
Sep 18 19:16:58.155151 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:58.155075    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sqcfr_281ef476-c8bd-4c7a-9357-5a7adb1f1ee7/dns-node-resolver/0.log"
Sep 18 19:16:58.911583 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:16:58.911554    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qv5lr_fcc6b30b-8557-41c1-93b4-50db142d69eb/node-ca/0.log"
Sep 18 19:17:00.829955 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:17:00.829908    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c66559bbb-jdc69_7ef4bddc-01ec-4ee6-9a48-8f2e81586749/kube-auth-proxy/0.log"
Sep 18 19:17:01.518491 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:17:01.518463    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g6kgh/perf-node-gather-daemonset-jc96p"
Sep 18 19:17:02.002351 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:17:02.002294    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qfvct_8780960a-d4de-43a1-86f6-a227b558013f/serve-healthcheck-canary/0.log"
Sep 18 19:17:02.629592 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:17:02.629558    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gc7jn_25e887b0-03c3-463e-8009-9e0970a083c6/kube-rbac-proxy/0.log"
Sep 18 19:17:02.651330 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:17:02.651304    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gc7jn_25e887b0-03c3-463e-8009-9e0970a083c6/exporter/0.log"
Sep 18 19:17:02.672741 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:17:02.672713    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gc7jn_25e887b0-03c3-463e-8009-9e0970a083c6/extractor/0.log"
Sep 18 19:17:05.890552 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:17:05.890521    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-64fbd675b6-5f5tg_c2e5dd2b-6fb6-483a-9d7c-08c0d6ca4f22/manager/0.log"
Sep 18 19:17:06.013958 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:17:06.013927    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c66559bbb-jdc69_7ef4bddc-01ec-4ee6-9a48-8f2e81586749/kube-auth-proxy/0.log"
Sep 18 19:17:06.316464 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:17:06.316423    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55bcff78dd-v2xdw_5a391624-5f5e-4cff-9532-6a6703ac9d86/manager/0.log"
Sep 18 19:17:06.337042 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:17:06.337017    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55bcff78dd-v2xdw_5a391624-5f5e-4cff-9532-6a6703ac9d86/copy-manifests/0.log"
Sep 18 19:17:07.680633 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:17:07.680597    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7cddfd44f8-mhnnj_2e96bcb4-8ab1-4777-93fc-03b873fe4442/manager/0.log"
Sep 18 19:17:08.342568 ip-10-0-130-118 kubenswrapper[2558]: I0918 19:17:08.342515    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c66559bbb-jdc69_7ef4bddc-01ec-4ee6-9a48-8f2e81586749/kube-auth-proxy/0.log"