Jul 26 15:16:38.831273 ip-10-0-139-110 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 26 15:16:38.831288 ip-10-0-139-110 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 26 15:16:38.831297 ip-10-0-139-110 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 26 15:16:38.831635 ip-10-0-139-110 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 26 15:16:49.022279 ip-10-0-139-110 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 26 15:16:49.022302 ip-10-0-139-110 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 61fc593507e1405b953d1c4e63d01fa5 --
Jul 26 15:18:35.590519 ip-10-0-139-110 systemd[1]: Starting Kubernetes Kubelet...
Jul 26 15:18:36.000215 ip-10-0-139-110 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 15:18:36.000215 ip-10-0-139-110 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 26 15:18:36.000215 ip-10-0-139-110 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 15:18:36.000215 ip-10-0-139-110 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 26 15:18:36.000215 ip-10-0-139-110 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 15:18:36.001128 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.001008    2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 26 15:18:36.006365 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006346    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 15:18:36.006365 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006362    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006368    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006373    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006376    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006380    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006383    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006386    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006388    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006391    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006393    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006396    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006399    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006402    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006404    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006407    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006409    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006412    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006415    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006417    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006420    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 15:18:36.006431 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006436    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006439    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006442    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006444    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006447    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006449    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006452    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006455    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006458    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006460    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006462    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006465    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006467    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006469    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006472    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006474    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006477    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006480    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006483    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006485    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006488    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 15:18:36.006893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006490    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006493    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006495    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006498    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006500    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006503    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006505    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006508    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006510    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006513    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006515    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006518    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006521    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006523    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006526    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006528    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006531    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006533    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006536    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006538    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 15:18:36.007434 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006540    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 15:18:36.007903 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006543    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 15:18:36.007903 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006547    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 15:18:36.007903 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006550    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 15:18:36.007903 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006553    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 15:18:36.007903 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006556    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 15:18:36.007903 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006559    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 15:18:36.007903 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006561    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 15:18:36.007903 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006564    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 15:18:36.007903 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006567    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 15:18:36.007903 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006569    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 15:18:36.007903 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006572    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 15:18:36.007903 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006575    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 15:18:36.007903 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006577    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 15:18:36.007903 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006580    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 15:18:36.007903 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006582    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 15:18:36.007903 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006585    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 15:18:36.007903 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006588    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 15:18:36.007903 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006591    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006595    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006598    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006601    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006604    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006607    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006610    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006614    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006617    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006620    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006623    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006626    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006629    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006631    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006634    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006636    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006639    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006642    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.006644    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 15:18:36.008354 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007358    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007366    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007369    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007373    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007376    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007378    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007381    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007384    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007388    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007391    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007393    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007396    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007399    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007402    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007405    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007407    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007410    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007413    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007415    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007418    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 15:18:36.008814 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007421    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007424    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007427    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007429    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007432    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007434    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007437    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007439    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007442    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007444    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007447    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007450    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007452    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007455    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007457    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007460    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007462    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007465    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007467    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007470    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 15:18:36.009430 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007472    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007475    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007477    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007480    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007483    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007486    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007489    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007493    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007496    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007498    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007501    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007503    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007505    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007508    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007511    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007513    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007516    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007518    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007521    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 15:18:36.009953 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007523    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007526    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007528    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007531    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007533    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007536    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007538    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007540    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007543    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007545    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007548    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007550    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007553    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007555    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007557    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007560    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007564    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007568    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007570    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007573    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 15:18:36.010419 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007576    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007579    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007581    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007584    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007586    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007589    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007591    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007594    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007597    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007599    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007603    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007606    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007609    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007612    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007614    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007616    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007619    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007622    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007624    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007627    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 15:18:36.010885 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007981    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007986    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007989    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007991    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007994    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007997    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.007999    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008002    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008006    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008008    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008011    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008013    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008016    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008019    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008023    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008025    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008028    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008031    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008034    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 15:18:36.011373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008036    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008040    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008042    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008045    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008048    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008050    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008053    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008055    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008058    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008061    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008064    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008067    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008069    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008072    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008074    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008077    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008079    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008081    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008084    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008086    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 15:18:36.011815 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008089    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008092    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008095    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008097    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008101    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008105    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008108    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008111    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008113    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008116    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008119    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008122    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008125    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008128    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008131    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008133    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008136    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008138    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008141    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 15:18:36.012296 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008143    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008146    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008149    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008151    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008154    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008156    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008158    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008161    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008163    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008167    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008170    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008173    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008175    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008178    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008181    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008184    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008186    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008189    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008196    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008212    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 15:18:36.012741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008215    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008217    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008220    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008223    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008226    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008229    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008232    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008234    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008238    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008241    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008243    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008246    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008248    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008251    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008254    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008256    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008258    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008261    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008263    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008265    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 15:18:36.013218 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008268    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008334    2559 flags.go:64] FLAG: --address="0.0.0.0"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008341    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008347    2559 flags.go:64] FLAG: --anonymous-auth="true"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008351    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008355    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008359    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008363    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008368    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008372    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008376    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008380    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008383    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008386    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008389    2559 flags.go:64] FLAG: --cgroup-root=""
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008392    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008395    2559 flags.go:64] FLAG: --client-ca-file=""
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008397    2559 flags.go:64] FLAG: --cloud-provider="external"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008400    2559 flags.go:64] FLAG: --cluster-dns="[]"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008404    2559 flags.go:64] FLAG: --cluster-domain=""
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008407    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008411    2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008414    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008417    2559 flags.go:64] FLAG: --container-log-max-files="5"
Jul 26 15:18:36.013699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008421    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008424    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008427    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008430    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008433    2559 flags.go:64] FLAG: --contention-profiling="false"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008436    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008439    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008442    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008444    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008448    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008451    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008453    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008456    2559 flags.go:64] FLAG: --enable-load-reader="false"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008459    2559 flags.go:64] FLAG: --enable-server="true"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008462    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008466    2559 flags.go:64] FLAG: --event-burst="100"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008469    2559 flags.go:64] FLAG: --event-qps="50"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008472    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008476    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008479    2559 flags.go:64] FLAG: --eviction-hard=""
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008482    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008486    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008489    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008492    2559 flags.go:64] FLAG: --eviction-soft=""
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008495    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 26 15:18:36.014309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008497    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008500    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008503    2559 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008506    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008508    2559 flags.go:64] FLAG: --fail-swap-on="true"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008511    2559 flags.go:64] FLAG: --feature-gates=""
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008519    2559 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008522    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008525    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008529    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008531    2559 flags.go:64] FLAG: --healthz-port="10248"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008534    2559 flags.go:64] FLAG: --help="false"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008537    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-139-110.ec2.internal"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008541    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008544    2559 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008546    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008550    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008553    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008556    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008560    2559 flags.go:64] FLAG: --image-service-endpoint=""
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008562    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008565    2559 flags.go:64] FLAG: --kube-api-burst="100"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008568    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008571    2559 flags.go:64] FLAG: --kube-api-qps="50"
Jul 26 15:18:36.014929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008574    2559 flags.go:64] FLAG: --kube-reserved=""
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008578    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008581    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008584    2559 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008586    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008589    2559 flags.go:64] FLAG: --lock-file=""
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008592    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008595    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008598    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008604    2559 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008606    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008609    2559 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008612    2559 flags.go:64] FLAG: --logging-format="text"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008615    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008618    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008621    2559 flags.go:64] FLAG: --manifest-url=""
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008624    2559 flags.go:64] FLAG: --manifest-url-header=""
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008628    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008631    2559 flags.go:64] FLAG: --max-open-files="1000000"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008635    2559 flags.go:64] FLAG: --max-pods="110"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008640    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008643    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008645    2559 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008648    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008651    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 26 15:18:36.015507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008654    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008657    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008663    2559 flags.go:64] FLAG: --node-status-max-images="50"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008667    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008669    2559 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008672    2559 flags.go:64] FLAG: --pod-cidr=""
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008675    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008681    2559 flags.go:64] FLAG: --pod-manifest-path=""
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008684    2559 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008688    2559 flags.go:64] FLAG: --pods-per-core="0"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008691    2559 flags.go:64] FLAG: --port="10250"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008694    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008697    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09f3f00940aeade2a"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008700    2559 flags.go:64] FLAG: --qos-reserved=""
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008703    2559 flags.go:64] FLAG: --read-only-port="10255"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008705    2559 flags.go:64] FLAG: --register-node="true"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008708    2559 flags.go:64] FLAG: --register-with-taints=""
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008711    2559 flags.go:64] FLAG: --registry-burst="10"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008714    2559 flags.go:64] FLAG: --registry-qps="5"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008717    2559 flags.go:64] FLAG: --reserved-cpus=""
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008720    2559 flags.go:64] FLAG: --reserved-memory=""
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008723    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008727    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008730    2559 flags.go:64] FLAG: --rotate-certificates="false"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008733    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 26 15:18:36.016113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008735    2559 flags.go:64] FLAG: --runonce="false"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008738    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008742    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008745    2559 flags.go:64] FLAG: --seccomp-default="false"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008748    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008751    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008754    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008757    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008760    2559 flags.go:64] FLAG: --storage-driver-password="root"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008762    2559 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008765    2559 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008768    2559 flags.go:64] FLAG: --storage-driver-user="root"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008771    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008774    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008776    2559 flags.go:64] FLAG: --system-cgroups=""
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008779    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008786    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008789    2559 flags.go:64] FLAG: --tls-cert-file=""
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008792    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008795    2559 flags.go:64] FLAG: --tls-min-version=""
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008798    2559 flags.go:64] FLAG: --tls-private-key-file=""
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008801    2559 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008803    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008813    2559 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008817    2559 flags.go:64] FLAG: --v="2"
Jul 26 15:18:36.016897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008821    2559 flags.go:64] FLAG: --version="false"
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008826    2559 flags.go:64] FLAG: --vmodule=""
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008830    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.008833    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008948    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008952    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008955    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008957    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008960    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008964    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008967    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008970    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008972    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008974    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008977    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008979    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008982    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008984    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008986    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008989    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 15:18:36.017813 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008991    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008993    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008996    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.008999    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009002    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009004    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009006    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009009    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009011    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009014    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009017    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009020    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009024    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009026    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009030    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009033    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009036    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009039    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009042    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 15:18:36.018389 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009044    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009047    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009050    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009053    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009055    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009058    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009060    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009063    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009065    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009067    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009070    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009072    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009074    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009077    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009079    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009081    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009084    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009087    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009089    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009092    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 15:18:36.018853 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009094    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009097    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009099    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009102    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009104    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009106    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009109    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009111    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009114    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009116    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009119    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009121    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009124    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009127    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009131    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009133    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009135    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009138    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009140    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 15:18:36.019395 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009143    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009145    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009148    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009150    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009152    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009154    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009157    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009159    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009162    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009164    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009167    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009170    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009172    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009175    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009177    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009179    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009181    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009184    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009186    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009189    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 15:18:36.019858 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009192    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 15:18:36.020365 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009196    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 15:18:36.020365 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009210    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 15:18:36.020365 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009212    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 15:18:36.020365 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.009215    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 15:18:36.020365 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.009223    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 15:18:36.020365 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.018462    2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 26 15:18:36.020365 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.018484    2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 26 15:18:36.020365 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.018501    2559 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 26 15:18:36.020365 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.018507    2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 26 15:18:36.020365 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018565    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 15:18:36.020365 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018570    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 15:18:36.020365 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018574    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 15:18:36.020365 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018577    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 15:18:36.020365 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018580    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 15:18:36.020365 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018582    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018585    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018588    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018591    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018594    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018596    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018599    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018602    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018604    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018608    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018613    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018616    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018619    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018622    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018625    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018627    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018630    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018632    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018635    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018637    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 15:18:36.020731 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018640    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018642    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018646    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018650    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018653    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018656    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018659    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018661    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018664    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018666    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018669    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018671    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018674    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018677    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018679    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018682    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018684    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018687    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018689    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 15:18:36.021241 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018692    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018694    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018697    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018700    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018702    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018705    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018707    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018710    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018713    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018715    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018718    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018720    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018722    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018725    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018729    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018732    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018735    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018739    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018742    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 15:18:36.021684 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018744    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018747    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018750    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018752    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018755    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018758    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018761    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018763    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018766    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018768    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018771    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018774    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018777    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018780    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018782    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018785    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018788    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018791    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018793    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018795    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 15:18:36.022135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018798    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 15:18:36.022652 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018801    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 15:18:36.022652 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018803    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 15:18:36.022652 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018806    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 15:18:36.022652 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018808    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 15:18:36.022652 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018811    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 15:18:36.022652 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018813    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 15:18:36.022652 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018815    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 15:18:36.022652 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018818    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 15:18:36.022652 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018820    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 15:18:36.022652 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018823    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 15:18:36.022652 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018826    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 15:18:36.022652 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018829    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 15:18:36.022652 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018832    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 15:18:36.022652 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018834    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 15:18:36.022652 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018837    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 15:18:36.023004 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.018842    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 15:18:36.023004 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018969    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 15:18:36.023004 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018974    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 15:18:36.023004 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018977    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 15:18:36.023004 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018980    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 15:18:36.023004 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018983    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 15:18:36.023004 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018985    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 15:18:36.023004 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018988    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 15:18:36.023004 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018991    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 15:18:36.023004 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018994    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 15:18:36.023004 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018996    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 15:18:36.023004 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.018999    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 15:18:36.023004 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019001    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 15:18:36.023004 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019005    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 15:18:36.023004 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019007    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019010    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019012    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019014    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019017    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019019    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019022    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019024    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019027    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019029    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019033    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019039    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019043    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019046    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019048    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019051    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019054    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019057    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019060    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 15:18:36.023429 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019063    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019065    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019068    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019071    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019073    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019075    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019078    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019080    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019083    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019085    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019088    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019090    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019092    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019096    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019098    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019101    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019103    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019106    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019108    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019111    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 15:18:36.023877 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019113    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019116    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019118    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019120    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019123    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019126    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019128    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019131    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019133    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019136    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019138    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019141    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019144    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019147    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019150    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019153    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019155    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019157    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019160    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 15:18:36.024468 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019162    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019164    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019167    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019169    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019172    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019174    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019176    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019180    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019184    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019186    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019189    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019192    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019195    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019211    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019217    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019220    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019222    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019226    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019229    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 15:18:36.025029 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019231    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 15:18:36.025493 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019234    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 15:18:36.025493 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019236    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 15:18:36.025493 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019239    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 15:18:36.025493 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019241    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 15:18:36.025493 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019244    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 15:18:36.025493 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019246    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 15:18:36.025493 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019249    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 15:18:36.025493 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:36.019251    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 15:18:36.025493 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.019256    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 15:18:36.025493 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.019380    2559 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 26 15:18:36.025493 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.023018    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 26 15:18:36.025493 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.024433    2559 server.go:1019] "Starting client certificate rotation"
Jul 26 15:18:36.025493 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.024558    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 26 15:18:36.025493 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.024626    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 26 15:18:36.027066 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.027049    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 26 15:18:36.028586 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.028570    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 26 15:18:36.039156 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.039138    2559 log.go:25] "Validated CRI v1 runtime API"
Jul 26 15:18:36.041255 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.041238    2559 log.go:25] "Validated CRI v1 image API"
Jul 26 15:18:36.044230 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.044217    2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 26 15:18:36.046587 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.046565    2559 fs.go:135] Filesystem UUIDs: map[4d723627-bfd5-4a3d-bc12-b18f5901107f:/dev/nvme0n1p3 4fb22200-afed-41ac-9e10-eb96a85ccaab:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 26 15:18:36.046661 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.046587    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 26 15:18:36.051524 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.051406    2559 manager.go:216] Machine: {Timestamp:2026-07-26 15:18:36.050185703 +0000 UTC m=+0.340017813 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3204284 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cee7b298c4288723fd9828c364338 SystemUUID:ec2cee7b-298c-4288-723f-d9828c364338 BootID:61fc5935-07e1-405b-953d-1c4e63d01fa5 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:69:13:f8:27:09 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:69:13:f8:27:09 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:1e:0e:95:3c:86 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 26 15:18:36.051524 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.051517    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 26 15:18:36.051685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.051628    2559 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 26 15:18:36.055652 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.055241    2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 26 15:18:36.056081 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.055660    2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-110.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 26 15:18:36.056867 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.056850    2559 topology_manager.go:138] "Creating topology manager with none policy"
Jul 26 15:18:36.056911 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.056877    2559 container_manager_linux.go:309] "Creating device plugin manager"
Jul 26 15:18:36.056911 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.056887    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 26 15:18:36.057618 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.057606    2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 26 15:18:36.058884 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.058874    2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 26 15:18:36.059447 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.059430    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 26 15:18:36.061560 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.061548    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 26 15:18:36.061706 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.061696    2559 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 26 15:18:36.061738 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.061727    2559 kubelet.go:486] "Attempting to sync node with API server"
Jul 26 15:18:36.061738 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.061736    2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 26 15:18:36.061786 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.061749    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 26 15:18:36.061786 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.061758    2559 kubelet.go:398] "Adding apiserver pod source"
Jul 26 15:18:36.061786 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.061766    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 26 15:18:36.062146 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.062135    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 26 15:18:36.062220 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.062152    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 26 15:18:36.064269 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.064248    2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 26 15:18:36.065981 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.065968    2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 26 15:18:36.066039 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.065997    2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 26 15:18:36.068172 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.068156    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 26 15:18:36.068225 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.068180    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 26 15:18:36.068225 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.068187    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 26 15:18:36.068225 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.068191    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 26 15:18:36.068225 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.068213    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 26 15:18:36.068225 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.068220    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 26 15:18:36.068354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.068228    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 26 15:18:36.068354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.068234    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 26 15:18:36.068354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.068241    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 26 15:18:36.068354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.068246    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 26 15:18:36.068354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.068254    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 26 15:18:36.068354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.068260    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 26 15:18:36.070729 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.070716    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 26 15:18:36.070729 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.070729    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 26 15:18:36.071046 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.071017    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 26 15:18:36.071102 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.071017    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-110.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 26 15:18:36.074018 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.073845    2559 server.go:1268] "Started kubelet"
Jul 26 15:18:36.074018 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.073938    2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 26 15:18:36.074107 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.074025    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 26 15:18:36.074107 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.074077    2559 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 26 15:18:36.074731 ip-10-0-139-110 systemd[1]: Started Kubernetes Kubelet.
Jul 26 15:18:36.075720 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.075698    2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-110.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 26 15:18:36.076533 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.076510    2559 server.go:310] "Adding debug handlers to kubelet server"
Jul 26 15:18:36.078251 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.078232    2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 26 15:18:36.079258 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.079230    2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 26 15:18:36.079581 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.079567    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8rp8z"
Jul 26 15:18:36.083894 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.082829    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-110.ec2.internal.18c5dfff918c321e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-110.ec2.internal,UID:ip-10-0-139-110.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-110.ec2.internal,},FirstTimestamp:2026-07-26 15:18:36.073808414 +0000 UTC m=+0.363640527,LastTimestamp:2026-07-26 15:18:36.073808414 +0000 UTC m=+0.363640527,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-110.ec2.internal,}"
Jul 26 15:18:36.083894 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.083884    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 26 15:18:36.084041 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.083898    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 26 15:18:36.084041 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.083979    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-110.ec2.internal\" not found"
Jul 26 15:18:36.084610 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.084591    2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 26 15:18:36.084700 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.084620    2559 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 26 15:18:36.084700 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.084636    2559 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 26 15:18:36.084813 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.084708    2559 reconstruct.go:97] "Volume reconstruction finished"
Jul 26 15:18:36.084813 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.084717    2559 reconciler.go:29] "Reconciler: start to sync state"
Jul 26 15:18:36.084910 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.084863    2559 factory.go:55] Registering systemd factory
Jul 26 15:18:36.084910 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.084901    2559 factory.go:223] Registration of the systemd container factory successfully
Jul 26 15:18:36.085312 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.085285    2559 factory.go:153] Registering CRI-O factory
Jul 26 15:18:36.085312 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.085300    2559 factory.go:223] Registration of the crio container factory successfully
Jul 26 15:18:36.085457 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.085356    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 26 15:18:36.085457 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.085383    2559 factory.go:103] Registering Raw factory
Jul 26 15:18:36.085457 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.085394    2559 manager.go:1196] Started watching for new ooms in manager
Jul 26 15:18:36.086012 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.085939    2559 manager.go:349] Starting recovery of all containers
Jul 26 15:18:36.087601 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.087555    2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-110.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 26 15:18:36.087708 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.087643    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 26 15:18:36.091018 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.090996    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8rp8z"
Jul 26 15:18:36.099889 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.099873    2559 manager.go:354] Recovery completed
Jul 26 15:18:36.104219 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.104206    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:18:36.106211 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.106195    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:18:36.106301 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.106230    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:18:36.106301 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.106241    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:18:36.106760 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.106744    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 26 15:18:36.106760 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.106755    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:18:36.106833 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.106761    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 26 15:18:36.106833 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.106778    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 26 15:18:36.107440 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.107421    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:18:36.107544 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.107454    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:18:36.107544 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.107469    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:18:36.108508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.108493    2559 policy_none.go:49] "None policy: Start"
Jul 26 15:18:36.108560 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.108511    2559 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 26 15:18:36.108560 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.108523    2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 26 15:18:36.109191 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.109113    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-110.ec2.internal.18c5dfff937aa2d0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-110.ec2.internal,UID:ip-10-0-139-110.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-110.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-110.ec2.internal,},FirstTimestamp:2026-07-26 15:18:36.106212048 +0000 UTC m=+0.396044167,LastTimestamp:2026-07-26 15:18:36.106212048 +0000 UTC m=+0.396044167,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-110.ec2.internal,}"
Jul 26 15:18:36.109528 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.109517    2559 policy_none.go:47] "Start"
Jul 26 15:18:36.147110 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.146893    2559 manager.go:337] "Starting Device Plugin manager"
Jul 26 15:18:36.147272 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.147145    2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 26 15:18:36.147272 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.147161    2559 server.go:87] "Starting device plugin registration server"
Jul 26 15:18:36.147487 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.147473    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 26 15:18:36.147543 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.147488    2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 26 15:18:36.147633 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.147614    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 26 15:18:36.147725 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.147711    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 26 15:18:36.147779 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.147727    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 26 15:18:36.148350 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.148329    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 26 15:18:36.148417 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.148389    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-110.ec2.internal\" not found"
Jul 26 15:18:36.185756 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.185723    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 26 15:18:36.192125 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.186986    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 26 15:18:36.192125 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.187035    2559 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 26 15:18:36.192125 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.187061    2559 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 26 15:18:36.192125 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.187117    2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 26 15:18:36.192125 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.189841    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 15:18:36.250848 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.250706    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:18:36.251907 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.251887    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:18:36.251985 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.251922    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:18:36.251985 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.251933    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:18:36.251985 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.251964    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-110.ec2.internal"
Jul 26 15:18:36.260722 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.260698    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-110.ec2.internal"
Jul 26 15:18:36.260811 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.260727    2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-110.ec2.internal\": node \"ip-10-0-139-110.ec2.internal\" not found"
Jul 26 15:18:36.278299 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.278268    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-110.ec2.internal\" not found"
Jul 26 15:18:36.287828 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.287797    2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-110.ec2.internal"]
Jul 26 15:18:36.287963 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.287877    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:18:36.289755 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.289736    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:18:36.289863 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.289774    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:18:36.289863 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.289789    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:18:36.292171 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.292154    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:18:36.292296 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.292267    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal"
Jul 26 15:18:36.292347 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.292310    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:18:36.293065 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.293048    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:18:36.293065 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.293061    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:18:36.293203 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.293080    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:18:36.293203 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.293095    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:18:36.293203 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.293082    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:18:36.293203 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.293184    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:18:36.295572 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.295555    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-110.ec2.internal"
Jul 26 15:18:36.295661 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.295593    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:18:36.297428 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.297410    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:18:36.297504 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.297437    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:18:36.297504 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.297447    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:18:36.321258 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.321237    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-110.ec2.internal\" not found" node="ip-10-0-139-110.ec2.internal"
Jul 26 15:18:36.326086 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.326069    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-110.ec2.internal\" not found" node="ip-10-0-139-110.ec2.internal"
Jul 26 15:18:36.379443 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.379409    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-110.ec2.internal\" not found"
Jul 26 15:18:36.386214 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.386185    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/58d8a889e41c1cf708108977adf7b177-config\") pod \"kube-apiserver-proxy-ip-10-0-139-110.ec2.internal\" (UID: \"58d8a889e41c1cf708108977adf7b177\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-110.ec2.internal"
Jul 26 15:18:36.386319 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.386244    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5432be730d7ab9428096bba07123643d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal\" (UID: \"5432be730d7ab9428096bba07123643d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal"
Jul 26 15:18:36.386319 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.386299    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5432be730d7ab9428096bba07123643d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal\" (UID: \"5432be730d7ab9428096bba07123643d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal"
Jul 26 15:18:36.479630 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.479586    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-110.ec2.internal\" not found"
Jul 26 15:18:36.486438 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.486409    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/58d8a889e41c1cf708108977adf7b177-config\") pod \"kube-apiserver-proxy-ip-10-0-139-110.ec2.internal\" (UID: \"58d8a889e41c1cf708108977adf7b177\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-110.ec2.internal"
Jul 26 15:18:36.486528 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.486449    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5432be730d7ab9428096bba07123643d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal\" (UID: \"5432be730d7ab9428096bba07123643d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal"
Jul 26 15:18:36.486528 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.486469    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5432be730d7ab9428096bba07123643d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal\" (UID: \"5432be730d7ab9428096bba07123643d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal"
Jul 26 15:18:36.486528 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.486472    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/58d8a889e41c1cf708108977adf7b177-config\") pod \"kube-apiserver-proxy-ip-10-0-139-110.ec2.internal\" (UID: \"58d8a889e41c1cf708108977adf7b177\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-110.ec2.internal"
Jul 26 15:18:36.486528 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.486501    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5432be730d7ab9428096bba07123643d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal\" (UID: \"5432be730d7ab9428096bba07123643d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal"
Jul 26 15:18:36.486917 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.486496    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5432be730d7ab9428096bba07123643d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal\" (UID: \"5432be730d7ab9428096bba07123643d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal"
Jul 26 15:18:36.580452 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.580357    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-110.ec2.internal\" not found"
Jul 26 15:18:36.625322 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.625280    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal"
Jul 26 15:18:36.630725 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.630502    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-110.ec2.internal"
Jul 26 15:18:36.680737 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.680693    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-110.ec2.internal\" not found"
Jul 26 15:18:36.785913 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.785874    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-110.ec2.internal\" not found"
Jul 26 15:18:36.886044 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:36.885959    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-110.ec2.internal\" not found"
Jul 26 15:18:36.908399 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.908367    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 15:18:36.989786 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:36.989739    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal"
Jul 26 15:18:37.003236 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.003202    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 26 15:18:37.005117 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.005100    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-110.ec2.internal"
Jul 26 15:18:37.013654 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.013624    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 26 15:18:37.023964 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.023942    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 26 15:18:37.024092 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.024072    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 26 15:18:37.024145 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.024122    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 26 15:18:37.062712 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.062677    2559 apiserver.go:52] "Watching apiserver"
Jul 26 15:18:37.074112 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.074082    2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 26 15:18:37.077331 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.077300    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-4d2ww","openshift-multus/multus-additional-cni-plugins-wnzvq","openshift-multus/network-metrics-daemon-p8dd2","kube-system/kube-apiserver-proxy-ip-10-0-139-110.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v","openshift-image-registry/node-ca-h8258","openshift-network-diagnostics/network-check-target-gztc2","openshift-network-operator/iptables-alerter-c2wft","openshift-ovn-kubernetes/ovnkube-node-qxgsx","kube-system/global-pull-secret-syncer-6g6lj","kube-system/konnectivity-agent-7wjc9","openshift-cluster-node-tuning-operator/tuned-vjlsk","openshift-dns/node-resolver-crzsx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal"]
Jul 26 15:18:37.080862 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.080843    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.082987 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.082965    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 26 15:18:37.083084 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.083060    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 26 15:18:37.083253 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.083239    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 26 15:18:37.083363 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.083285    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 26 15:18:37.083444 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.083426    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-k8c96\""
Jul 26 15:18:37.083615 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.083593    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.084010 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.083993    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 26 15:18:37.085415 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.085398    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 26 15:18:37.085483 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.085431    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 26 15:18:37.085612 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.085599    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bh7ll\""
Jul 26 15:18:37.086431 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.086410    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:37.086520 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.086479    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:18:37.089151 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.089133    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.090694 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.090676    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-system-cni-dir\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.090779 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.090699    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-host-run-netns\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.090779 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.090728    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/33a7f331-84f0-43c3-86fb-981f0fc567b9-multus-daemon-config\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.090779 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.090745    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-os-release\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.091033 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.090786    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffp7z\" (UniqueName: \"kubernetes.io/projected/63512a39-e727-488a-96ea-e93a866aad7d-kube-api-access-ffp7z\") pod \"network-metrics-daemon-p8dd2\" (UID: \"63512a39-e727-488a-96ea-e93a866aad7d\") " pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:37.091033 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.090832    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-multus-cni-dir\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.091033 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.090858    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 26 15:18:37.091033 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.090864    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/33a7f331-84f0-43c3-86fb-981f0fc567b9-cni-binary-copy\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.091033 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.090924    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-multus-socket-dir-parent\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.091033 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.090951    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-etc-kubernetes\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.091033 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.090978    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 26 15:18:37.091033 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091028    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-88brf\""
Jul 26 15:18:37.091450 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.090979    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 26 15:18:37.091450 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.090978    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-host-run-k8s-cni-cncf-io\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.091450 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091194    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.091450 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091230    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-os-release\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.091450 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091328    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-host-var-lib-cni-bin\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.091450 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091358    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd9lh\" (UniqueName: \"kubernetes.io/projected/33a7f331-84f0-43c3-86fb-981f0fc567b9-kube-api-access-fd9lh\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.091450 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091384    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-cni-binary-copy\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.091450 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091408    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-cnibin\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.091450 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091433    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-host-var-lib-cni-multus\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.091784 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091457    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-host-var-lib-kubelet\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.091784 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091480    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-hostroot\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.091784 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091506    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-system-cni-dir\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.091784 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091542    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-multus-conf-dir\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.091784 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091573    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.091784 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091643    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtxvv\" (UniqueName: \"kubernetes.io/projected/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-kube-api-access-vtxvv\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.091784 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091671    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs\") pod \"network-metrics-daemon-p8dd2\" (UID: \"63512a39-e727-488a-96ea-e93a866aad7d\") " pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:37.091784 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091704    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-host-run-multus-certs\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.091784 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091730    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-cnibin\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.091784 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091757    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.092161 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.091993    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h8258"
Jul 26 15:18:37.093769 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.093747    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 26 15:18:37.093769 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.093765    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 26 15:18:37.094005 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.093775    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 26 15:18:37.094005 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.093854    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ktrxr\""
Jul 26 15:18:37.095068 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.095053    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:37.095146 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.095130    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:18:37.095388 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.095373    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 26 15:18:37.097896 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.097881    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c2wft"
Jul 26 15:18:37.098591 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.098551    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-25 15:13:36 +0000 UTC" deadline="2028-02-25 22:44:30.54118438 +0000 UTC"
Jul 26 15:18:37.098660 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.098591    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13903h25m53.442596483s"
Jul 26 15:18:37.099894 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.099877    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 26 15:18:37.099973 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.099902    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 26 15:18:37.099973 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.099928    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 26 15:18:37.100125 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.100112    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4z5t2\""
Jul 26 15:18:37.100969 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.100945    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.102960 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.102944    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 26 15:18:37.103060 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.102976    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 26 15:18:37.103372 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.103350    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 26 15:18:37.103372 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.103368    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-j9czv\""
Jul 26 15:18:37.103587 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.103446    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 26 15:18:37.103587 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.103456    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 26 15:18:37.103587 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.103445    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 26 15:18:37.103781 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.103767    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:37.103847 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.103832    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:18:37.106700 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.106681    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7wjc9"
Jul 26 15:18:37.109166 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.109144    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 26 15:18:37.109438 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.109420    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 26 15:18:37.109628 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.109603    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wbqwn\""
Jul 26 15:18:37.110262 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.110242    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.111726 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.111708    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 26 15:18:37.111936 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.111923    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 26 15:18:37.112002 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.111989    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-k896s\""
Jul 26 15:18:37.112880 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.112863    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-crzsx"
Jul 26 15:18:37.114742 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.114724    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 26 15:18:37.114856 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.114834    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wtv6j\""
Jul 26 15:18:37.115055 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.115040    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 26 15:18:37.120194 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.120176    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qs775"
Jul 26 15:18:37.128772 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.128747    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qs775"
Jul 26 15:18:37.185864 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.185839    2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 26 15:18:37.192247 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192220    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-cni-netd\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.192397 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192254    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-os-release\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.192397 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192277    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd9lh\" (UniqueName: \"kubernetes.io/projected/33a7f331-84f0-43c3-86fb-981f0fc567b9-kube-api-access-fd9lh\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.192397 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192319    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-cni-binary-copy\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.192627 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192396    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-os-release\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.192627 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192425    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5645fd38-99a5-4fa7-bf50-bf044c0444b6-dbus\") pod \"global-pull-secret-syncer-6g6lj\" (UID: \"5645fd38-99a5-4fa7-bf50-bf044c0444b6\") " pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:37.192627 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192457    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-sys\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.192627 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192483    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-var-lib-openvswitch\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.192627 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192507    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-ovnkube-config\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.192627 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192531    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d6d718b7-9f87-444a-af05-964167a2f945-agent-certs\") pod \"konnectivity-agent-7wjc9\" (UID: \"d6d718b7-9f87-444a-af05-964167a2f945\") " pod="kube-system/konnectivity-agent-7wjc9"
Jul 26 15:18:37.192627 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192553    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg8lk\" (UniqueName: \"kubernetes.io/projected/02874aa1-a6f9-47fa-af6f-373b4f475a65-kube-api-access-cg8lk\") pod \"node-ca-h8258\" (UID: \"02874aa1-a6f9-47fa-af6f-373b4f475a65\") " pod="openshift-image-registry/node-ca-h8258"
Jul 26 15:18:37.192627 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192575    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret\") pod \"global-pull-secret-syncer-6g6lj\" (UID: \"5645fd38-99a5-4fa7-bf50-bf044c0444b6\") " pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:37.192627 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192611    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-run-openvswitch\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.193059 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192637    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-node-log\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.193059 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192658    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9-hosts-file\") pod \"node-resolver-crzsx\" (UID: \"72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9\") " pod="openshift-dns/node-resolver-crzsx"
Jul 26 15:18:37.193059 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192690    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9-tmp-dir\") pod \"node-resolver-crzsx\" (UID: \"72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9\") " pod="openshift-dns/node-resolver-crzsx"
Jul 26 15:18:37.193059 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192732    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-host-var-lib-cni-multus\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.193059 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192760    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-host-var-lib-kubelet\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.193059 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192811    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-host-var-lib-kubelet\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.193059 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192853    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-host-var-lib-cni-multus\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.193059 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192894    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed19b225-8b95-43c5-9e23-c7da28caec86-etc-selinux\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.193059 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192931    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-etc-modprobe-d\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.193059 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192960    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd53803c-1587-47ba-be2d-3e44c0eb892c-host-slash\") pod \"iptables-alerter-c2wft\" (UID: \"dd53803c-1587-47ba-be2d-3e44c0eb892c\") " pod="openshift-network-operator/iptables-alerter-c2wft"
Jul 26 15:18:37.193059 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.192987    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed19b225-8b95-43c5-9e23-c7da28caec86-sys-fs\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.193059 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193005    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-host\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.193059 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193031    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8slvp\" (UniqueName: \"kubernetes.io/projected/dd53803c-1587-47ba-be2d-3e44c0eb892c-kube-api-access-8slvp\") pod \"iptables-alerter-c2wft\" (UID: \"dd53803c-1587-47ba-be2d-3e44c0eb892c\") " pod="openshift-network-operator/iptables-alerter-c2wft"
Jul 26 15:18:37.193059 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193054    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-run-netns\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.193773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193075    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-cnibin\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.193773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193079    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-cni-binary-copy\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.193773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193094    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed19b225-8b95-43c5-9e23-c7da28caec86-socket-dir\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.193773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193120    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s28q\" (UniqueName: \"kubernetes.io/projected/5105e897-4d5f-4256-b9d5-5f656c670215-kube-api-access-9s28q\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.193773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193147    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-system-cni-dir\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.193773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193158    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-cnibin\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.193773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193171    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed19b225-8b95-43c5-9e23-c7da28caec86-kubelet-dir\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.193773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193195    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-ovn-node-metrics-cert\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.193773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193222    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-multus-cni-dir\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.193773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193240    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-system-cni-dir\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.193773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193263    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-multus-socket-dir-parent\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.193773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193415    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-multus-socket-dir-parent\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.193773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193417    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxgvd\" (UniqueName: \"kubernetes.io/projected/ed19b225-8b95-43c5-9e23-c7da28caec86-kube-api-access-hxgvd\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.193773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193454    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-run\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.193773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193475    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-multus-cni-dir\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.193773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193482    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d6d718b7-9f87-444a-af05-964167a2f945-konnectivity-ca\") pod \"konnectivity-agent-7wjc9\" (UID: \"d6d718b7-9f87-444a-af05-964167a2f945\") " pod="kube-system/konnectivity-agent-7wjc9"
Jul 26 15:18:37.193773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193509    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-host-run-k8s-cni-cncf-io\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.194768 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193549    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed19b225-8b95-43c5-9e23-c7da28caec86-device-dir\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.194768 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193579    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-host-run-k8s-cni-cncf-io\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.194768 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193595    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-etc-sysctl-d\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.194768 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193624    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5105e897-4d5f-4256-b9d5-5f656c670215-etc-tuned\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.194768 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193641    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-run-ovn-kubernetes\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.194768 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193662    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-host-var-lib-cni-bin\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.194768 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193687    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-cni-bin\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.194768 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193704    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wb7t\" (UniqueName: \"kubernetes.io/projected/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-kube-api-access-7wb7t\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.194768 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193703    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-host-var-lib-cni-bin\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.194768 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193722    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-cnibin\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.194768 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193746    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-env-overrides\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.194768 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193763    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-hostroot\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.194768 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193767    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-cnibin\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.194768 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193778    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-system-cni-dir\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.194768 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193807    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-lib-modules\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.194768 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193820    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-hostroot\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.194768 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193809    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-system-cni-dir\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.195685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193834    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dd53803c-1587-47ba-be2d-3e44c0eb892c-iptables-alerter-script\") pod \"iptables-alerter-c2wft\" (UID: \"dd53803c-1587-47ba-be2d-3e44c0eb892c\") " pod="openshift-network-operator/iptables-alerter-c2wft"
Jul 26 15:18:37.195685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193870    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-slash\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.195685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193885    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-log-socket\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.195685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193907    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.195685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193934    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdtrq\" (UniqueName: \"kubernetes.io/projected/72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9-kube-api-access-jdtrq\") pod \"node-resolver-crzsx\" (UID: \"72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9\") " pod="openshift-dns/node-resolver-crzsx"
Jul 26 15:18:37.195685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193960    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-multus-conf-dir\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.195685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.193983    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.195685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.194007    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtxvv\" (UniqueName: \"kubernetes.io/projected/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-kube-api-access-vtxvv\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.195685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.194033    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs\") pod \"network-metrics-daemon-p8dd2\" (UID: \"63512a39-e727-488a-96ea-e93a866aad7d\") " pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:37.195685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.194069    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-multus-conf-dir\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.195685 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.194194    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:18:37.195685 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.194317    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs podName:63512a39-e727-488a-96ea-e93a866aad7d nodeName:}" failed. No retries permitted until 2026-07-26 15:18:37.694276391 +0000 UTC m=+1.984108489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs") pod "network-metrics-daemon-p8dd2" (UID: "63512a39-e727-488a-96ea-e93a866aad7d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:18:37.195685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.194058    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-etc-sysconfig\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.195685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.194503    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-etc-sysctl-conf\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.195685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.194531    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q8vl\" (UniqueName: \"kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl\") pod \"network-check-target-gztc2\" (UID: \"b003fa80-7770-4f06-9917-926f86feebf4\") " pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:37.195685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.194584    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-kubelet\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.196222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.194689    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-host-run-multus-certs\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.196222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.194843    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-host-run-multus-certs\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.196222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.194820    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.196222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.194901    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-var-lib-kubelet\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.196222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.194930    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-etc-openvswitch\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.196222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.194956    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-run-ovn\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.196222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.194990    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-ovnkube-script-lib\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.196222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195006    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.196222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195033    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-host-run-netns\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.196222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195061    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/33a7f331-84f0-43c3-86fb-981f0fc567b9-multus-daemon-config\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.196222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195093    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-os-release\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.196222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195120    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffp7z\" (UniqueName: \"kubernetes.io/projected/63512a39-e727-488a-96ea-e93a866aad7d-kube-api-access-ffp7z\") pod \"network-metrics-daemon-p8dd2\" (UID: \"63512a39-e727-488a-96ea-e93a866aad7d\") " pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:37.196222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195125    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-host-run-netns\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.196222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195146    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02874aa1-a6f9-47fa-af6f-373b4f475a65-host\") pod \"node-ca-h8258\" (UID: \"02874aa1-a6f9-47fa-af6f-373b4f475a65\") " pod="openshift-image-registry/node-ca-h8258"
Jul 26 15:18:37.196222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195251    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-os-release\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.196222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195315    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/02874aa1-a6f9-47fa-af6f-373b4f475a65-serviceca\") pod \"node-ca-h8258\" (UID: \"02874aa1-a6f9-47fa-af6f-373b4f475a65\") " pod="openshift-image-registry/node-ca-h8258"
Jul 26 15:18:37.196222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195359    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5105e897-4d5f-4256-b9d5-5f656c670215-tmp\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.196774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195394    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/33a7f331-84f0-43c3-86fb-981f0fc567b9-cni-binary-copy\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.196774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195427    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-etc-kubernetes\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.196774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195523    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33a7f331-84f0-43c3-86fb-981f0fc567b9-etc-kubernetes\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.196774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195543    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed19b225-8b95-43c5-9e23-c7da28caec86-registration-dir\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.196774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195533    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.196774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195566    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-etc-kubernetes\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.196774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195582    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-etc-systemd\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.196774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195624    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.196774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195675    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5645fd38-99a5-4fa7-bf50-bf044c0444b6-kubelet-config\") pod \"global-pull-secret-syncer-6g6lj\" (UID: \"5645fd38-99a5-4fa7-bf50-bf044c0444b6\") " pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:37.196774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195701    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-systemd-units\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.196774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195730    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-run-systemd\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.196774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195795    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/33a7f331-84f0-43c3-86fb-981f0fc567b9-multus-daemon-config\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.196774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195811    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.196774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.195867    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/33a7f331-84f0-43c3-86fb-981f0fc567b9-cni-binary-copy\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.206811 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.206790    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 26 15:18:37.210301 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.210238    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtxvv\" (UniqueName: \"kubernetes.io/projected/c95dc61e-9fe1-40b4-85cf-edb31a0c5607-kube-api-access-vtxvv\") pod \"multus-additional-cni-plugins-wnzvq\" (UID: \"c95dc61e-9fe1-40b4-85cf-edb31a0c5607\") " pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.210404 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.210242    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd9lh\" (UniqueName: \"kubernetes.io/projected/33a7f331-84f0-43c3-86fb-981f0fc567b9-kube-api-access-fd9lh\") pod \"multus-4d2ww\" (UID: \"33a7f331-84f0-43c3-86fb-981f0fc567b9\") " pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.210446 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.210423    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffp7z\" (UniqueName: \"kubernetes.io/projected/63512a39-e727-488a-96ea-e93a866aad7d-kube-api-access-ffp7z\") pod \"network-metrics-daemon-p8dd2\" (UID: \"63512a39-e727-488a-96ea-e93a866aad7d\") " pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:37.264177 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:37.264128    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58d8a889e41c1cf708108977adf7b177.slice/crio-ef29b1e2f152228bfcec5e6caa1073d556ea1b15ccb913b8ceb69b897a702fee WatchSource:0}: Error finding container ef29b1e2f152228bfcec5e6caa1073d556ea1b15ccb913b8ceb69b897a702fee: Status 404 returned error can't find the container with id ef29b1e2f152228bfcec5e6caa1073d556ea1b15ccb913b8ceb69b897a702fee
Jul 26 15:18:37.264706 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:37.264684    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5432be730d7ab9428096bba07123643d.slice/crio-1e83a52e94fd314f21e570b7e9394474791d763a0a82a86bbb319196de7c077a WatchSource:0}: Error finding container 1e83a52e94fd314f21e570b7e9394474791d763a0a82a86bbb319196de7c077a: Status 404 returned error can't find the container with id 1e83a52e94fd314f21e570b7e9394474791d763a0a82a86bbb319196de7c077a
Jul 26 15:18:37.272038 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.271579    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 15:18:37.296001 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.295965    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9-hosts-file\") pod \"node-resolver-crzsx\" (UID: \"72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9\") " pod="openshift-dns/node-resolver-crzsx"
Jul 26 15:18:37.296242 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296013    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9-tmp-dir\") pod \"node-resolver-crzsx\" (UID: \"72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9\") " pod="openshift-dns/node-resolver-crzsx"
Jul 26 15:18:37.296242 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296040    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed19b225-8b95-43c5-9e23-c7da28caec86-etc-selinux\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.296242 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296067    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-etc-modprobe-d\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.296242 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296081    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9-hosts-file\") pod \"node-resolver-crzsx\" (UID: \"72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9\") " pod="openshift-dns/node-resolver-crzsx"
Jul 26 15:18:37.296242 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296091    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd53803c-1587-47ba-be2d-3e44c0eb892c-host-slash\") pod \"iptables-alerter-c2wft\" (UID: \"dd53803c-1587-47ba-be2d-3e44c0eb892c\") " pod="openshift-network-operator/iptables-alerter-c2wft"
Jul 26 15:18:37.296242 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296137    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd53803c-1587-47ba-be2d-3e44c0eb892c-host-slash\") pod \"iptables-alerter-c2wft\" (UID: \"dd53803c-1587-47ba-be2d-3e44c0eb892c\") " pod="openshift-network-operator/iptables-alerter-c2wft"
Jul 26 15:18:37.296242 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296140    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed19b225-8b95-43c5-9e23-c7da28caec86-sys-fs\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.296242 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296167    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-host\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.296242 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296182    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed19b225-8b95-43c5-9e23-c7da28caec86-etc-selinux\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.296242 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296194    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8slvp\" (UniqueName: \"kubernetes.io/projected/dd53803c-1587-47ba-be2d-3e44c0eb892c-kube-api-access-8slvp\") pod \"iptables-alerter-c2wft\" (UID: \"dd53803c-1587-47ba-be2d-3e44c0eb892c\") " pod="openshift-network-operator/iptables-alerter-c2wft"
Jul 26 15:18:37.296242 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296201    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed19b225-8b95-43c5-9e23-c7da28caec86-sys-fs\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.296664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296254    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-host\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.296664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296250    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-etc-modprobe-d\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.296664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296344    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-run-netns\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.296664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296377    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed19b225-8b95-43c5-9e23-c7da28caec86-socket-dir\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.296664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296384    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-run-netns\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.296664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296402    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s28q\" (UniqueName: \"kubernetes.io/projected/5105e897-4d5f-4256-b9d5-5f656c670215-kube-api-access-9s28q\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.296664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296480    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed19b225-8b95-43c5-9e23-c7da28caec86-kubelet-dir\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.296664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296504    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-ovn-node-metrics-cert\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.296664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296522    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxgvd\" (UniqueName: \"kubernetes.io/projected/ed19b225-8b95-43c5-9e23-c7da28caec86-kube-api-access-hxgvd\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.296664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296520    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed19b225-8b95-43c5-9e23-c7da28caec86-kubelet-dir\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.296664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296539    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-run\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.296664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296587    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-run\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.296664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296587    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed19b225-8b95-43c5-9e23-c7da28caec86-socket-dir\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.296664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296648    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d6d718b7-9f87-444a-af05-964167a2f945-konnectivity-ca\") pod \"konnectivity-agent-7wjc9\" (UID: \"d6d718b7-9f87-444a-af05-964167a2f945\") " pod="kube-system/konnectivity-agent-7wjc9"
Jul 26 15:18:37.297368 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296684    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed19b225-8b95-43c5-9e23-c7da28caec86-device-dir\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.297368 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296713    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-etc-sysctl-d\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.297368 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296741    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5105e897-4d5f-4256-b9d5-5f656c670215-etc-tuned\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.297368 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296760    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed19b225-8b95-43c5-9e23-c7da28caec86-device-dir\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.297368 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296766    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-run-ovn-kubernetes\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.297368 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296802    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-cni-bin\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.297368 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296832    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wb7t\" (UniqueName: \"kubernetes.io/projected/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-kube-api-access-7wb7t\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.297368 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296804    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-run-ovn-kubernetes\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.297368 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296853    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-etc-sysctl-d\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.297368 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296863    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-env-overrides\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.297368 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296891    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-lib-modules\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.297368 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296917    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dd53803c-1587-47ba-be2d-3e44c0eb892c-iptables-alerter-script\") pod \"iptables-alerter-c2wft\" (UID: \"dd53803c-1587-47ba-be2d-3e44c0eb892c\") " pod="openshift-network-operator/iptables-alerter-c2wft"
Jul 26 15:18:37.297368 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296837    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-cni-bin\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.297368 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296940    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-slash\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.297368 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296972    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-slash\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.297368 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296970    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9-tmp-dir\") pod \"node-resolver-crzsx\" (UID: \"72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9\") " pod="openshift-dns/node-resolver-crzsx"
Jul 26 15:18:37.297368 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297010    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-log-socket\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.298189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.296977    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-log-socket\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.298189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297104    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.298189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297132    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-lib-modules\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.298189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297134    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdtrq\" (UniqueName: \"kubernetes.io/projected/72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9-kube-api-access-jdtrq\") pod \"node-resolver-crzsx\" (UID: \"72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9\") " pod="openshift-dns/node-resolver-crzsx"
Jul 26 15:18:37.298189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297205    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-etc-sysconfig\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.298189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297235    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-etc-sysctl-conf\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.298189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297284    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q8vl\" (UniqueName: \"kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl\") pod \"network-check-target-gztc2\" (UID: \"b003fa80-7770-4f06-9917-926f86feebf4\") " pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:37.298189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297304    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-etc-sysconfig\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.298189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297311    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-kubelet\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.298189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297249    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.298189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297350    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d6d718b7-9f87-444a-af05-964167a2f945-konnectivity-ca\") pod \"konnectivity-agent-7wjc9\" (UID: \"d6d718b7-9f87-444a-af05-964167a2f945\") " pod="kube-system/konnectivity-agent-7wjc9"
Jul 26 15:18:37.298189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297400    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-kubelet\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.298189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297444    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-var-lib-kubelet\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.298189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297455    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-etc-sysctl-conf\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.298189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297492    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-etc-openvswitch\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.298189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297496    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-var-lib-kubelet\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.298189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297497    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-env-overrides\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.299052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297543    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-run-ovn\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.299052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297567    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dd53803c-1587-47ba-be2d-3e44c0eb892c-iptables-alerter-script\") pod \"iptables-alerter-c2wft\" (UID: \"dd53803c-1587-47ba-be2d-3e44c0eb892c\") " pod="openshift-network-operator/iptables-alerter-c2wft"
Jul 26 15:18:37.299052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297543    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-etc-openvswitch\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.299052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297572    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-ovnkube-script-lib\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.299052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297588    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-run-ovn\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.299052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297629    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02874aa1-a6f9-47fa-af6f-373b4f475a65-host\") pod \"node-ca-h8258\" (UID: \"02874aa1-a6f9-47fa-af6f-373b4f475a65\") " pod="openshift-image-registry/node-ca-h8258"
Jul 26 15:18:37.299052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297655    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/02874aa1-a6f9-47fa-af6f-373b4f475a65-serviceca\") pod \"node-ca-h8258\" (UID: \"02874aa1-a6f9-47fa-af6f-373b4f475a65\") " pod="openshift-image-registry/node-ca-h8258"
Jul 26 15:18:37.299052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297678    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5105e897-4d5f-4256-b9d5-5f656c670215-tmp\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.299052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297681    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02874aa1-a6f9-47fa-af6f-373b4f475a65-host\") pod \"node-ca-h8258\" (UID: \"02874aa1-a6f9-47fa-af6f-373b4f475a65\") " pod="openshift-image-registry/node-ca-h8258"
Jul 26 15:18:37.299052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297706    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed19b225-8b95-43c5-9e23-c7da28caec86-registration-dir\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.299052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297732    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-etc-kubernetes\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.299052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297757    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-etc-systemd\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.299052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297784    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5645fd38-99a5-4fa7-bf50-bf044c0444b6-kubelet-config\") pod \"global-pull-secret-syncer-6g6lj\" (UID: \"5645fd38-99a5-4fa7-bf50-bf044c0444b6\") " pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:37.299052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297805    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-etc-kubernetes\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.299052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297811    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-systemd-units\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.299052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297857    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-run-systemd\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.299052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297874    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-etc-systemd\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.299052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297897    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-cni-netd\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.299746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297917    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-systemd-units\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.299746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297926    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5645fd38-99a5-4fa7-bf50-bf044c0444b6-dbus\") pod \"global-pull-secret-syncer-6g6lj\" (UID: \"5645fd38-99a5-4fa7-bf50-bf044c0444b6\") " pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:37.299746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297951    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-sys\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.299746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297973    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5645fd38-99a5-4fa7-bf50-bf044c0444b6-kubelet-config\") pod \"global-pull-secret-syncer-6g6lj\" (UID: \"5645fd38-99a5-4fa7-bf50-bf044c0444b6\") " pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:37.299746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.297980    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-var-lib-openvswitch\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.299746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.298021    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-ovnkube-config\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.299746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.298053    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d6d718b7-9f87-444a-af05-964167a2f945-agent-certs\") pod \"konnectivity-agent-7wjc9\" (UID: \"d6d718b7-9f87-444a-af05-964167a2f945\") " pod="kube-system/konnectivity-agent-7wjc9"
Jul 26 15:18:37.299746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.298082    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5645fd38-99a5-4fa7-bf50-bf044c0444b6-dbus\") pod \"global-pull-secret-syncer-6g6lj\" (UID: \"5645fd38-99a5-4fa7-bf50-bf044c0444b6\") " pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:37.299746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.298083    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg8lk\" (UniqueName: \"kubernetes.io/projected/02874aa1-a6f9-47fa-af6f-373b4f475a65-kube-api-access-cg8lk\") pod \"node-ca-h8258\" (UID: \"02874aa1-a6f9-47fa-af6f-373b4f475a65\") " pod="openshift-image-registry/node-ca-h8258"
Jul 26 15:18:37.299746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.298120    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret\") pod \"global-pull-secret-syncer-6g6lj\" (UID: \"5645fd38-99a5-4fa7-bf50-bf044c0444b6\") " pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:37.299746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.298130    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-ovnkube-script-lib\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.299746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.298136    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/02874aa1-a6f9-47fa-af6f-373b4f475a65-serviceca\") pod \"node-ca-h8258\" (UID: \"02874aa1-a6f9-47fa-af6f-373b4f475a65\") " pod="openshift-image-registry/node-ca-h8258"
Jul 26 15:18:37.299746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.298022    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-var-lib-openvswitch\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.299746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.298288    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-run-openvswitch\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.299746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.298310    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-host-cni-netd\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.299746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.298316    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-node-log\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.299746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.298345    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5105e897-4d5f-4256-b9d5-5f656c670215-sys\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.299746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.298354    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-run-openvswitch\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.300358 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.298411    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed19b225-8b95-43c5-9e23-c7da28caec86-registration-dir\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.300358 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.298469    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-run-systemd\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.300358 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.298520    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-node-log\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.300358 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.298542    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:18:37.300358 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.298604    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret podName:5645fd38-99a5-4fa7-bf50-bf044c0444b6 nodeName:}" failed. No retries permitted until 2026-07-26 15:18:37.798584034 +0000 UTC m=+2.088416152 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret") pod "global-pull-secret-syncer-6g6lj" (UID: "5645fd38-99a5-4fa7-bf50-bf044c0444b6") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:18:37.300358 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.298833    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-ovnkube-config\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.300358 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.299733    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5105e897-4d5f-4256-b9d5-5f656c670215-etc-tuned\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.300358 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.299824    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-ovn-node-metrics-cert\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.300358 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.299857    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5105e897-4d5f-4256-b9d5-5f656c670215-tmp\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.300724 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.300708    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d6d718b7-9f87-444a-af05-964167a2f945-agent-certs\") pod \"konnectivity-agent-7wjc9\" (UID: \"d6d718b7-9f87-444a-af05-964167a2f945\") " pod="kube-system/konnectivity-agent-7wjc9"
Jul 26 15:18:37.303524 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.303497    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:18:37.303638 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.303529    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:18:37.303638 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.303545    2559 projected.go:196] Error preparing data for projected volume kube-api-access-5q8vl for pod openshift-network-diagnostics/network-check-target-gztc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:18:37.303638 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.303619    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl podName:b003fa80-7770-4f06-9917-926f86feebf4 nodeName:}" failed. No retries permitted until 2026-07-26 15:18:37.803598842 +0000 UTC m=+2.093430942 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5q8vl" (UniqueName: "kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl") pod "network-check-target-gztc2" (UID: "b003fa80-7770-4f06-9917-926f86feebf4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:18:37.304623 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.304595    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s28q\" (UniqueName: \"kubernetes.io/projected/5105e897-4d5f-4256-b9d5-5f656c670215-kube-api-access-9s28q\") pod \"tuned-vjlsk\" (UID: \"5105e897-4d5f-4256-b9d5-5f656c670215\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.304791 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.304767    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxgvd\" (UniqueName: \"kubernetes.io/projected/ed19b225-8b95-43c5-9e23-c7da28caec86-kube-api-access-hxgvd\") pod \"aws-ebs-csi-driver-node-szx2v\" (UID: \"ed19b225-8b95-43c5-9e23-c7da28caec86\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.304847 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.304829    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8slvp\" (UniqueName: \"kubernetes.io/projected/dd53803c-1587-47ba-be2d-3e44c0eb892c-kube-api-access-8slvp\") pod \"iptables-alerter-c2wft\" (UID: \"dd53803c-1587-47ba-be2d-3e44c0eb892c\") " pod="openshift-network-operator/iptables-alerter-c2wft"
Jul 26 15:18:37.305150 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.305133    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdtrq\" (UniqueName: \"kubernetes.io/projected/72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9-kube-api-access-jdtrq\") pod \"node-resolver-crzsx\" (UID: \"72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9\") " pod="openshift-dns/node-resolver-crzsx"
Jul 26 15:18:37.305524 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.305508    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wb7t\" (UniqueName: \"kubernetes.io/projected/3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9-kube-api-access-7wb7t\") pod \"ovnkube-node-qxgsx\" (UID: \"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.306358 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.306335    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg8lk\" (UniqueName: \"kubernetes.io/projected/02874aa1-a6f9-47fa-af6f-373b4f475a65-kube-api-access-cg8lk\") pod \"node-ca-h8258\" (UID: \"02874aa1-a6f9-47fa-af6f-373b4f475a65\") " pod="openshift-image-registry/node-ca-h8258"
Jul 26 15:18:37.405969 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.405880    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4d2ww"
Jul 26 15:18:37.416638 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:37.416609    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33a7f331_84f0_43c3_86fb_981f0fc567b9.slice/crio-3de8f6d63d4415756179c404460c860231f6b8872bb76e97d6a349e003a47077 WatchSource:0}: Error finding container 3de8f6d63d4415756179c404460c860231f6b8872bb76e97d6a349e003a47077: Status 404 returned error can't find the container with id 3de8f6d63d4415756179c404460c860231f6b8872bb76e97d6a349e003a47077
Jul 26 15:18:37.417202 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.417176    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wnzvq"
Jul 26 15:18:37.426173 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:37.426149    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc95dc61e_9fe1_40b4_85cf_edb31a0c5607.slice/crio-0f1a803a940cc3aef9e4efb2a5d3397b5fa0aca1b6fead8e1a5c063e2ca1ee59 WatchSource:0}: Error finding container 0f1a803a940cc3aef9e4efb2a5d3397b5fa0aca1b6fead8e1a5c063e2ca1ee59: Status 404 returned error can't find the container with id 0f1a803a940cc3aef9e4efb2a5d3397b5fa0aca1b6fead8e1a5c063e2ca1ee59
Jul 26 15:18:37.432456 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.432430    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v"
Jul 26 15:18:37.441854 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:37.441823    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded19b225_8b95_43c5_9e23_c7da28caec86.slice/crio-557e70c8ba5c47887e3f65a9112d74246331d245403b530803c1f84b44f0b0b1 WatchSource:0}: Error finding container 557e70c8ba5c47887e3f65a9112d74246331d245403b530803c1f84b44f0b0b1: Status 404 returned error can't find the container with id 557e70c8ba5c47887e3f65a9112d74246331d245403b530803c1f84b44f0b0b1
Jul 26 15:18:37.444977 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.444711    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h8258"
Jul 26 15:18:37.453051 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:37.453028    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02874aa1_a6f9_47fa_af6f_373b4f475a65.slice/crio-b93d82d8a13f483a219c14147bfeb47482d0800e3b80722af8b456841b93caa3 WatchSource:0}: Error finding container b93d82d8a13f483a219c14147bfeb47482d0800e3b80722af8b456841b93caa3: Status 404 returned error can't find the container with id b93d82d8a13f483a219c14147bfeb47482d0800e3b80722af8b456841b93caa3
Jul 26 15:18:37.457409 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.457386    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c2wft"
Jul 26 15:18:37.465972 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:37.465943    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd53803c_1587_47ba_be2d_3e44c0eb892c.slice/crio-94b8cc3ef1a413115f78e1fa1fa7b6c922c15f5a3dcf3bb4346c93591b07d4b2 WatchSource:0}: Error finding container 94b8cc3ef1a413115f78e1fa1fa7b6c922c15f5a3dcf3bb4346c93591b07d4b2: Status 404 returned error can't find the container with id 94b8cc3ef1a413115f78e1fa1fa7b6c922c15f5a3dcf3bb4346c93591b07d4b2
Jul 26 15:18:37.475712 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.475694    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:18:37.482941 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.482917    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7wjc9"
Jul 26 15:18:37.484658 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:37.484628    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ba37c3c_76dc_4d63_8c8d_ad6fef4964d9.slice/crio-40cd4e6e04b982a5b768587abfa5eb379dc4834609387b9e91e7d03f334e42f4 WatchSource:0}: Error finding container 40cd4e6e04b982a5b768587abfa5eb379dc4834609387b9e91e7d03f334e42f4: Status 404 returned error can't find the container with id 40cd4e6e04b982a5b768587abfa5eb379dc4834609387b9e91e7d03f334e42f4
Jul 26 15:18:37.492485 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:37.492461    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6d718b7_9f87_444a_af05_964167a2f945.slice/crio-525e250ee54ad35b288108862788392938292b5ba2d72f154b9cf7caf1e95b89 WatchSource:0}: Error finding container 525e250ee54ad35b288108862788392938292b5ba2d72f154b9cf7caf1e95b89: Status 404 returned error can't find the container with id 525e250ee54ad35b288108862788392938292b5ba2d72f154b9cf7caf1e95b89
Jul 26 15:18:37.504157 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.504131    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vjlsk"
Jul 26 15:18:37.507988 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.507961    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-crzsx"
Jul 26 15:18:37.514893 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.514875    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 15:18:37.514893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:37.514877    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5105e897_4d5f_4256_b9d5_5f656c670215.slice/crio-60d5a130c61a2dc0493532f4323dff7ca7f4e5cfb604b764b88074d2f8a8a6f6 WatchSource:0}: Error finding container 60d5a130c61a2dc0493532f4323dff7ca7f4e5cfb604b764b88074d2f8a8a6f6: Status 404 returned error can't find the container with id 60d5a130c61a2dc0493532f4323dff7ca7f4e5cfb604b764b88074d2f8a8a6f6
Jul 26 15:18:37.518671 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:18:37.518648    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72ff0d6f_05ae_4ce7_87b4_794a0d2c7bc9.slice/crio-1bebd006c00ab37fb3ab8405dcd01ff812097a5e51b2bfa3c4d6246b998192ac WatchSource:0}: Error finding container 1bebd006c00ab37fb3ab8405dcd01ff812097a5e51b2bfa3c4d6246b998192ac: Status 404 returned error can't find the container with id 1bebd006c00ab37fb3ab8405dcd01ff812097a5e51b2bfa3c4d6246b998192ac
Jul 26 15:18:37.595376 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.595345    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 15:18:37.701345 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.701244    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs\") pod \"network-metrics-daemon-p8dd2\" (UID: \"63512a39-e727-488a-96ea-e93a866aad7d\") " pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:37.701500 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.701396    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:18:37.701500 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.701462    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs podName:63512a39-e727-488a-96ea-e93a866aad7d nodeName:}" failed. No retries permitted until 2026-07-26 15:18:38.701447258 +0000 UTC m=+2.991279361 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs") pod "network-metrics-daemon-p8dd2" (UID: "63512a39-e727-488a-96ea-e93a866aad7d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:18:37.801778 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.801728    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret\") pod \"global-pull-secret-syncer-6g6lj\" (UID: \"5645fd38-99a5-4fa7-bf50-bf044c0444b6\") " pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:37.801954 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.801916    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:18:37.802014 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.801992    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret podName:5645fd38-99a5-4fa7-bf50-bf044c0444b6 nodeName:}" failed. No retries permitted until 2026-07-26 15:18:38.801976703 +0000 UTC m=+3.091808800 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret") pod "global-pull-secret-syncer-6g6lj" (UID: "5645fd38-99a5-4fa7-bf50-bf044c0444b6") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:18:37.902891 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:37.902848    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q8vl\" (UniqueName: \"kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl\") pod \"network-check-target-gztc2\" (UID: \"b003fa80-7770-4f06-9917-926f86feebf4\") " pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:37.903101 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.903083    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:18:37.903179 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.903113    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:18:37.903179 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.903127    2559 projected.go:196] Error preparing data for projected volume kube-api-access-5q8vl for pod openshift-network-diagnostics/network-check-target-gztc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:18:37.903323 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:37.903209    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl podName:b003fa80-7770-4f06-9917-926f86feebf4 nodeName:}" failed. No retries permitted until 2026-07-26 15:18:38.90318873 +0000 UTC m=+3.193020831 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5q8vl" (UniqueName: "kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl") pod "network-check-target-gztc2" (UID: "b003fa80-7770-4f06-9917-926f86feebf4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:18:38.035011 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.034881    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 15:18:38.132052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.132005    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 15:13:37 +0000 UTC" deadline="2027-12-25 09:46:15.98193996 +0000 UTC"
Jul 26 15:18:38.132052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.132050    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12402h27m37.849895466s"
Jul 26 15:18:38.188007 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.187974    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:38.188271 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:38.188138    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:18:38.188271 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.188157    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:38.188474 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:38.188294    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:18:38.202201 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.202126    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c2wft" event={"ID":"dd53803c-1587-47ba-be2d-3e44c0eb892c","Type":"ContainerStarted","Data":"94b8cc3ef1a413115f78e1fa1fa7b6c922c15f5a3dcf3bb4346c93591b07d4b2"}
Jul 26 15:18:38.207790 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.207749    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h8258" event={"ID":"02874aa1-a6f9-47fa-af6f-373b4f475a65","Type":"ContainerStarted","Data":"b93d82d8a13f483a219c14147bfeb47482d0800e3b80722af8b456841b93caa3"}
Jul 26 15:18:38.215957 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.215918    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnzvq" event={"ID":"c95dc61e-9fe1-40b4-85cf-edb31a0c5607","Type":"ContainerStarted","Data":"0f1a803a940cc3aef9e4efb2a5d3397b5fa0aca1b6fead8e1a5c063e2ca1ee59"}
Jul 26 15:18:38.235551 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.235505    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4d2ww" event={"ID":"33a7f331-84f0-43c3-86fb-981f0fc567b9","Type":"ContainerStarted","Data":"3de8f6d63d4415756179c404460c860231f6b8872bb76e97d6a349e003a47077"}
Jul 26 15:18:38.271570 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.271324    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal" event={"ID":"5432be730d7ab9428096bba07123643d","Type":"ContainerStarted","Data":"1e83a52e94fd314f21e570b7e9394474791d763a0a82a86bbb319196de7c077a"}
Jul 26 15:18:38.277750 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.277707    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-crzsx" event={"ID":"72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9","Type":"ContainerStarted","Data":"1bebd006c00ab37fb3ab8405dcd01ff812097a5e51b2bfa3c4d6246b998192ac"}
Jul 26 15:18:38.302584 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.302484    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx" event={"ID":"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9","Type":"ContainerStarted","Data":"40cd4e6e04b982a5b768587abfa5eb379dc4834609387b9e91e7d03f334e42f4"}
Jul 26 15:18:38.334249 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.334152    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v" event={"ID":"ed19b225-8b95-43c5-9e23-c7da28caec86","Type":"ContainerStarted","Data":"557e70c8ba5c47887e3f65a9112d74246331d245403b530803c1f84b44f0b0b1"}
Jul 26 15:18:38.359495 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.359451    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-110.ec2.internal" event={"ID":"58d8a889e41c1cf708108977adf7b177","Type":"ContainerStarted","Data":"ef29b1e2f152228bfcec5e6caa1073d556ea1b15ccb913b8ceb69b897a702fee"}
Jul 26 15:18:38.364166 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.364124    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vjlsk" event={"ID":"5105e897-4d5f-4256-b9d5-5f656c670215","Type":"ContainerStarted","Data":"60d5a130c61a2dc0493532f4323dff7ca7f4e5cfb604b764b88074d2f8a8a6f6"}
Jul 26 15:18:38.379430 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.379389    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7wjc9" event={"ID":"d6d718b7-9f87-444a-af05-964167a2f945","Type":"ContainerStarted","Data":"525e250ee54ad35b288108862788392938292b5ba2d72f154b9cf7caf1e95b89"}
Jul 26 15:18:38.713484 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.713044    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs\") pod \"network-metrics-daemon-p8dd2\" (UID: \"63512a39-e727-488a-96ea-e93a866aad7d\") " pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:38.713484 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:38.713207    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:18:38.713484 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:38.713295    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs podName:63512a39-e727-488a-96ea-e93a866aad7d nodeName:}" failed. No retries permitted until 2026-07-26 15:18:40.713274597 +0000 UTC m=+5.003106712 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs") pod "network-metrics-daemon-p8dd2" (UID: "63512a39-e727-488a-96ea-e93a866aad7d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:18:38.814576 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.814536    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret\") pod \"global-pull-secret-syncer-6g6lj\" (UID: \"5645fd38-99a5-4fa7-bf50-bf044c0444b6\") " pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:38.814772 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:38.814693    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:18:38.814772 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:38.814759    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret podName:5645fd38-99a5-4fa7-bf50-bf044c0444b6 nodeName:}" failed. No retries permitted until 2026-07-26 15:18:40.814739691 +0000 UTC m=+5.104571806 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret") pod "global-pull-secret-syncer-6g6lj" (UID: "5645fd38-99a5-4fa7-bf50-bf044c0444b6") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:18:38.917778 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:38.917582    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q8vl\" (UniqueName: \"kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl\") pod \"network-check-target-gztc2\" (UID: \"b003fa80-7770-4f06-9917-926f86feebf4\") " pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:38.917983 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:38.917813    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:18:38.917983 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:38.917835    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:18:38.917983 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:38.917848    2559 projected.go:196] Error preparing data for projected volume kube-api-access-5q8vl for pod openshift-network-diagnostics/network-check-target-gztc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:18:38.917983 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:38.917911    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl podName:b003fa80-7770-4f06-9917-926f86feebf4 nodeName:}" failed. No retries permitted until 2026-07-26 15:18:40.917892133 +0000 UTC m=+5.207724249 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5q8vl" (UniqueName: "kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl") pod "network-check-target-gztc2" (UID: "b003fa80-7770-4f06-9917-926f86feebf4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:18:39.132591 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:39.132463    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 15:13:37 +0000 UTC" deadline="2028-04-12 21:25:46.77858349 +0000 UTC"
Jul 26 15:18:39.132591 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:39.132535    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15030h7m7.646054175s"
Jul 26 15:18:39.187815 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:39.187776    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:39.188031 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:39.187928    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:18:40.063467 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:40.063433    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 15:18:40.190134 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:40.190101    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:40.190594 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:40.190255    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:18:40.190712 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:40.190689    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:40.190822 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:40.190799    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:18:40.732678 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:40.732580    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs\") pod \"network-metrics-daemon-p8dd2\" (UID: \"63512a39-e727-488a-96ea-e93a866aad7d\") " pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:40.732854 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:40.732802    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:18:40.732917 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:40.732870    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs podName:63512a39-e727-488a-96ea-e93a866aad7d nodeName:}" failed. No retries permitted until 2026-07-26 15:18:44.73285093 +0000 UTC m=+9.022683045 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs") pod "network-metrics-daemon-p8dd2" (UID: "63512a39-e727-488a-96ea-e93a866aad7d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:18:40.833960 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:40.833905    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret\") pod \"global-pull-secret-syncer-6g6lj\" (UID: \"5645fd38-99a5-4fa7-bf50-bf044c0444b6\") " pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:40.834202 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:40.834148    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:18:40.834296 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:40.834221    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret podName:5645fd38-99a5-4fa7-bf50-bf044c0444b6 nodeName:}" failed. No retries permitted until 2026-07-26 15:18:44.834201862 +0000 UTC m=+9.124033964 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret") pod "global-pull-secret-syncer-6g6lj" (UID: "5645fd38-99a5-4fa7-bf50-bf044c0444b6") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:18:40.934464 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:40.934421    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q8vl\" (UniqueName: \"kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl\") pod \"network-check-target-gztc2\" (UID: \"b003fa80-7770-4f06-9917-926f86feebf4\") " pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:40.934711 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:40.934692    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:18:40.934807 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:40.934719    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:18:40.934807 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:40.934733    2559 projected.go:196] Error preparing data for projected volume kube-api-access-5q8vl for pod openshift-network-diagnostics/network-check-target-gztc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:18:40.934914 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:40.934814    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl podName:b003fa80-7770-4f06-9917-926f86feebf4 nodeName:}" failed. No retries permitted until 2026-07-26 15:18:44.934786434 +0000 UTC m=+9.224618538 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5q8vl" (UniqueName: "kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl") pod "network-check-target-gztc2" (UID: "b003fa80-7770-4f06-9917-926f86feebf4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:18:41.188398 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:41.188305    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:41.188568 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:41.188462    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:18:42.188199 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:42.187593    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:42.188199 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:42.187775    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:18:42.189067 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:42.188913    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:42.189067 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:42.189029    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:18:43.187775 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:43.187726    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:43.188030 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:43.187894    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:18:44.188282 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:44.188244    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:44.188282 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:44.188256    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:44.188796 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:44.188381    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:18:44.188796 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:44.188529    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:18:44.771073 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:44.770391    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs\") pod \"network-metrics-daemon-p8dd2\" (UID: \"63512a39-e727-488a-96ea-e93a866aad7d\") " pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:44.771073 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:44.770624    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:18:44.771073 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:44.770695    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs podName:63512a39-e727-488a-96ea-e93a866aad7d nodeName:}" failed. No retries permitted until 2026-07-26 15:18:52.770673691 +0000 UTC m=+17.060505814 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs") pod "network-metrics-daemon-p8dd2" (UID: "63512a39-e727-488a-96ea-e93a866aad7d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:18:44.871263 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:44.871194    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret\") pod \"global-pull-secret-syncer-6g6lj\" (UID: \"5645fd38-99a5-4fa7-bf50-bf044c0444b6\") " pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:44.871466 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:44.871372    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:18:44.871466 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:44.871443    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret podName:5645fd38-99a5-4fa7-bf50-bf044c0444b6 nodeName:}" failed. No retries permitted until 2026-07-26 15:18:52.871420653 +0000 UTC m=+17.161252750 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret") pod "global-pull-secret-syncer-6g6lj" (UID: "5645fd38-99a5-4fa7-bf50-bf044c0444b6") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:18:44.972343 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:44.972303    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q8vl\" (UniqueName: \"kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl\") pod \"network-check-target-gztc2\" (UID: \"b003fa80-7770-4f06-9917-926f86feebf4\") " pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:44.972521 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:44.972498    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:18:44.972588 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:44.972525    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:18:44.972588 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:44.972537    2559 projected.go:196] Error preparing data for projected volume kube-api-access-5q8vl for pod openshift-network-diagnostics/network-check-target-gztc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:18:44.972656 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:44.972596    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl podName:b003fa80-7770-4f06-9917-926f86feebf4 nodeName:}" failed. No retries permitted until 2026-07-26 15:18:52.972581494 +0000 UTC m=+17.262413593 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5q8vl" (UniqueName: "kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl") pod "network-check-target-gztc2" (UID: "b003fa80-7770-4f06-9917-926f86feebf4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:18:45.188076 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:45.187987    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:45.188279 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:45.188132    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:18:46.189204 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:46.188565    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:46.189204 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:46.188699    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:18:46.189204 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:46.189068    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:46.189204 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:46.189160    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:18:47.187285 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:47.187250    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:47.187496 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:47.187390    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:18:48.187997 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:48.187960    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:48.188358 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:48.188067    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:18:48.188358 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:48.188123    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:48.188358 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:48.188226    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:18:49.187934 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:49.187897    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:49.188112 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:49.188021    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:18:50.187803 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:50.187763    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:50.187803 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:50.187803    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:50.188037 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:50.187898    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:18:50.188037 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:50.188018    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:18:51.187558 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:51.187515    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:51.188024 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:51.187663    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:18:52.187479 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:52.187434    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:52.187672 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:52.187586    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:18:52.187672 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:52.187634    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:52.188076 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:52.187731    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:18:52.831619 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:52.831574    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs\") pod \"network-metrics-daemon-p8dd2\" (UID: \"63512a39-e727-488a-96ea-e93a866aad7d\") " pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:52.831839 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:52.831698    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:18:52.831839 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:52.831767    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs podName:63512a39-e727-488a-96ea-e93a866aad7d nodeName:}" failed. No retries permitted until 2026-07-26 15:19:08.831750091 +0000 UTC m=+33.121582206 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs") pod "network-metrics-daemon-p8dd2" (UID: "63512a39-e727-488a-96ea-e93a866aad7d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:18:52.932563 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:52.932523    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret\") pod \"global-pull-secret-syncer-6g6lj\" (UID: \"5645fd38-99a5-4fa7-bf50-bf044c0444b6\") " pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:52.932732 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:52.932710    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:18:52.932814 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:52.932792    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret podName:5645fd38-99a5-4fa7-bf50-bf044c0444b6 nodeName:}" failed. No retries permitted until 2026-07-26 15:19:08.932772541 +0000 UTC m=+33.222604644 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret") pod "global-pull-secret-syncer-6g6lj" (UID: "5645fd38-99a5-4fa7-bf50-bf044c0444b6") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:18:53.033548 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:53.033502    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q8vl\" (UniqueName: \"kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl\") pod \"network-check-target-gztc2\" (UID: \"b003fa80-7770-4f06-9917-926f86feebf4\") " pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:53.033732 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:53.033657    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:18:53.033732 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:53.033677    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:18:53.033732 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:53.033686    2559 projected.go:196] Error preparing data for projected volume kube-api-access-5q8vl for pod openshift-network-diagnostics/network-check-target-gztc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:18:53.033893 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:53.033744    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl podName:b003fa80-7770-4f06-9917-926f86feebf4 nodeName:}" failed. No retries permitted until 2026-07-26 15:19:09.033725363 +0000 UTC m=+33.323557461 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5q8vl" (UniqueName: "kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl") pod "network-check-target-gztc2" (UID: "b003fa80-7770-4f06-9917-926f86feebf4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:18:53.188101 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:53.188009    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:53.188561 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:53.188134    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:18:54.187815 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:54.187772    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:54.187815 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:54.187819    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:54.188044 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:54.187926    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:18:54.188107 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:54.188054    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:18:55.187579 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:55.187536    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:55.188029 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:55.187656    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:18:56.188840 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:56.188800    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:56.189302 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:56.188913    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:18:56.189302 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:56.189010    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:56.189302 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:56.189123    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:18:57.188132 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:57.188088    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:57.188318 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:57.188236    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:18:58.188135 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.187761    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:18:58.188743 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.187761    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:18:58.188743 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:58.188266    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:18:58.188743 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:58.188303    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:18:58.442673 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.442588    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h8258" event={"ID":"02874aa1-a6f9-47fa-af6f-373b4f475a65","Type":"ContainerStarted","Data":"c1baedebd57829ddec4fc135664accf7924c067f633c7fd0c2993d190c26874c"}
Jul 26 15:18:58.444159 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.444128    2559 generic.go:356] "Generic (PLEG): container finished" podID="c95dc61e-9fe1-40b4-85cf-edb31a0c5607" containerID="095ea1e366cf4f5686fcc01b9e8e8408a3bdfcdb3eb2fb17ed13f416221cdc88" exitCode=0
Jul 26 15:18:58.444322 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.444223    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnzvq" event={"ID":"c95dc61e-9fe1-40b4-85cf-edb31a0c5607","Type":"ContainerDied","Data":"095ea1e366cf4f5686fcc01b9e8e8408a3bdfcdb3eb2fb17ed13f416221cdc88"}
Jul 26 15:18:58.445913 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.445876    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4d2ww" event={"ID":"33a7f331-84f0-43c3-86fb-981f0fc567b9","Type":"ContainerStarted","Data":"6e6ef3e4bb9b67a69f1ae5199dda2e9e78015f4edd93bcb673203dc1f8c6afb7"}
Jul 26 15:18:58.447998 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.447973    2559 generic.go:356] "Generic (PLEG): container finished" podID="5432be730d7ab9428096bba07123643d" containerID="ab99328e5371194024f95190f2846a1ee93eb1fe4a997dc35555c8fad14d1a4f" exitCode=0
Jul 26 15:18:58.448100 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.448055    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal" event={"ID":"5432be730d7ab9428096bba07123643d","Type":"ContainerDied","Data":"ab99328e5371194024f95190f2846a1ee93eb1fe4a997dc35555c8fad14d1a4f"}
Jul 26 15:18:58.449677 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.449652    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-crzsx" event={"ID":"72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9","Type":"ContainerStarted","Data":"a1102348da85b3d91fa3fe51945d64f57ddf09d3a1ccfc2fb5e5decbf208d4f7"}
Jul 26 15:18:58.452866 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.452842    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx" event={"ID":"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9","Type":"ContainerStarted","Data":"61635019567b9c6b6b09949081c3aed4e81b63e59fb70432cb6ddb0c1ac9e658"}
Jul 26 15:18:58.452961 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.452920    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx" event={"ID":"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9","Type":"ContainerStarted","Data":"418b217d9f661d233dbb187b5d89ace5110a891a4f7eddd9c22e48652286717d"}
Jul 26 15:18:58.453049 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.452969    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx" event={"ID":"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9","Type":"ContainerStarted","Data":"402ee1b86207ca63eddb5794765bbc8f6143f33e70d0b0693d878ef82b9f4a20"}
Jul 26 15:18:58.453049 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.453023    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx" event={"ID":"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9","Type":"ContainerStarted","Data":"b3bbe0bef07c7d018a698e981cdc68347ddacfc68bb27ac63fec5f62034c343a"}
Jul 26 15:18:58.453137 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.453067    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx" event={"ID":"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9","Type":"ContainerStarted","Data":"7a9140fa7a075d1e9682e5b7d0b1513b97df9ac6062788a122ec1172405e71d6"}
Jul 26 15:18:58.453137 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.453114    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx" event={"ID":"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9","Type":"ContainerStarted","Data":"543bbb68fd6340b86c581c325c3179bfc20630eebf221acbb7a1355140a86be2"}
Jul 26 15:18:58.454360 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.454334    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v" event={"ID":"ed19b225-8b95-43c5-9e23-c7da28caec86","Type":"ContainerStarted","Data":"1ebb4311a1229819e09aa7e1af8c6c9a43547a79243ecd5387e14901a15c3b4b"}
Jul 26 15:18:58.455827 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.455763    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-110.ec2.internal" event={"ID":"58d8a889e41c1cf708108977adf7b177","Type":"ContainerStarted","Data":"3be4cbbe42178ffcf324844e2ecb078a26ca83138e8974ae8d208b217b79b989"}
Jul 26 15:18:58.457311 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.457291    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vjlsk" event={"ID":"5105e897-4d5f-4256-b9d5-5f656c670215","Type":"ContainerStarted","Data":"744949c9bd88187da40255c5b4cfae16edc072e454fdcc132067d7ed9c598e0c"}
Jul 26 15:18:58.458686 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.458664    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7wjc9" event={"ID":"d6d718b7-9f87-444a-af05-964167a2f945","Type":"ContainerStarted","Data":"d7b94b4edfa79128c50a7579582cc14711e5bc38edf3827600b8f3918cfb9df7"}
Jul 26 15:18:58.477365 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.477317    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-h8258" podStartSLOduration=1.473333773 podStartE2EDuration="21.47729816s" totalImagesPullingTime="20.003964387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:18:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:18:58.462372767 +0000 UTC m=+22.752204892" watchObservedRunningTime="2026-07-26 15:18:58.47729816 +0000 UTC m=+22.767130284"
Jul 26 15:18:58.493880 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.493827    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4d2ww" podStartSLOduration=1.404348809 podStartE2EDuration="21.49381487s" totalImagesPullingTime="20.089466061s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:18:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:18:58.493605272 +0000 UTC m=+22.783437391" watchObservedRunningTime="2026-07-26 15:18:58.49381487 +0000 UTC m=+22.783646988"
Jul 26 15:18:58.507086 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.507017    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-110.ec2.internal" podStartSLOduration=21.506996964 podStartE2EDuration="21.506996964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:18:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:18:58.50644579 +0000 UTC m=+22.796277909" watchObservedRunningTime="2026-07-26 15:18:58.506996964 +0000 UTC m=+22.796829083"
Jul 26 15:18:58.567271 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.567227    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7wjc9" podStartSLOduration=1.6104706100000001 podStartE2EDuration="21.567195022s" totalImagesPullingTime="19.956724412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:18:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:18:58.544190094 +0000 UTC m=+22.834022210" watchObservedRunningTime="2026-07-26 15:18:58.567195022 +0000 UTC m=+22.857027138"
Jul 26 15:18:58.567381 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.567306    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vjlsk" podStartSLOduration=1.632382014 podStartE2EDuration="21.56730124s" totalImagesPullingTime="19.934919226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:18:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:18:58.562277916 +0000 UTC m=+22.852110034" watchObservedRunningTime="2026-07-26 15:18:58.56730124 +0000 UTC m=+22.857133358"
Jul 26 15:18:58.580242 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.579369    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-crzsx" podStartSLOduration=1.634506684 podStartE2EDuration="21.579348823s" totalImagesPullingTime="19.944842139s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:18:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:18:58.577627506 +0000 UTC m=+22.867461784" watchObservedRunningTime="2026-07-26 15:18:58.579348823 +0000 UTC m=+22.869180934"
Jul 26 15:18:58.621522 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:58.621492    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:18:59.076731 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:59.076698    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 26 15:18:59.161941 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:59.161849    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-26T15:18:59.07672296Z","UUID":"4ce04ec8-e37f-4252-97a1-64d7f5f751ed","Handler":null,"Name":"","Endpoint":""}
Jul 26 15:18:59.163242 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:59.163225    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 26 15:18:59.163318 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:59.163260    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 26 15:18:59.187456 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:59.187429    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:18:59.187660 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:18:59.187602    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:18:59.463136 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:59.463044    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c2wft" event={"ID":"dd53803c-1587-47ba-be2d-3e44c0eb892c","Type":"ContainerStarted","Data":"a55695498c65240d38e130ba1881ad04a665d048b538cc54bf118962ad2ea867"}
Jul 26 15:18:59.465001 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:59.464975    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal" event={"ID":"5432be730d7ab9428096bba07123643d","Type":"ContainerStarted","Data":"a3ed82e5779098751f21bccd55e94b407acf2545e2b2ed7816137ff6dd2df376"}
Jul 26 15:18:59.467105 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:59.467078    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v" event={"ID":"ed19b225-8b95-43c5-9e23-c7da28caec86","Type":"ContainerStarted","Data":"85f282d5db7ca80ee1e5eaa484a26a2f1da6af4eb3fe670d2754939265858554"}
Jul 26 15:18:59.478281 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:59.478196    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-c2wft" podStartSLOduration=2.488462324 podStartE2EDuration="22.478176493s" totalImagesPullingTime="19.989714169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:18:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:18:59.477906852 +0000 UTC m=+23.767738969" watchObservedRunningTime="2026-07-26 15:18:59.478176493 +0000 UTC m=+23.768008612"
Jul 26 15:18:59.492417 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:59.492364    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-110.ec2.internal" podStartSLOduration=22.492348944 podStartE2EDuration="22.492348944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:18:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:18:59.492018142 +0000 UTC m=+23.781850262" watchObservedRunningTime="2026-07-26 15:18:59.492348944 +0000 UTC m=+23.782181061"
Jul 26 15:18:59.890846 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:59.890756    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7wjc9"
Jul 26 15:18:59.891400 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:18:59.891376    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7wjc9"
Jul 26 15:19:00.188015 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:00.187896    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:00.188015 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:00.187918    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:00.188307 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:00.188042    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:19:00.188307 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:00.188145    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:19:00.473808 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:00.473719    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx" event={"ID":"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9","Type":"ContainerStarted","Data":"5fb2bd597c9ef3825c31a0f9f4df474f64a1c373fc29e886afb33afede2d86d3"}
Jul 26 15:19:01.187302 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:01.187259    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:01.187502 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:01.187420    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:19:01.482154 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:01.482064    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v" event={"ID":"ed19b225-8b95-43c5-9e23-c7da28caec86","Type":"ContainerStarted","Data":"c8a532e52d66cac20ad52491b34cef07bec73627eb5419077d1d46b2286c875e"}
Jul 26 15:19:01.482154 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:01.482109    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 26 15:19:01.500216 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:01.500139    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-szx2v" podStartSLOduration=1.957338952 podStartE2EDuration="24.500119978s" totalImagesPullingTime="22.542781026s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:18:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:19:01.499493279 +0000 UTC m=+25.789325397" watchObservedRunningTime="2026-07-26 15:19:01.500119978 +0000 UTC m=+25.789952098"
Jul 26 15:19:01.569311 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:01.569274    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:01.852887 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:01.852806    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:02.107133 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:02.107055    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:02.187407 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:02.187372    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:02.187593 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:02.187373    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:02.187593 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:02.187568    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:19:02.187784 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:02.187731    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:19:02.224918 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:02.224860    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:02.325736 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:02.325691    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:02.448593 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:02.448512    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:02.526985 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:02.526955    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:02.629965 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:02.629868    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:03.187898 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:03.187669    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:03.188043 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:03.187932    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:19:03.488795 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:03.488717    2559 generic.go:356] "Generic (PLEG): container finished" podID="c95dc61e-9fe1-40b4-85cf-edb31a0c5607" containerID="ee7b9fc54d768bbd363fa069d3e1d9ff279e6727b096bbb64b7e18dfe249a342" exitCode=0
Jul 26 15:19:03.488968 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:03.488806    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnzvq" event={"ID":"c95dc61e-9fe1-40b4-85cf-edb31a0c5607","Type":"ContainerDied","Data":"ee7b9fc54d768bbd363fa069d3e1d9ff279e6727b096bbb64b7e18dfe249a342"}
Jul 26 15:19:03.492746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:03.492700    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx" event={"ID":"3ba37c3c-76dc-4d63-8c8d-ad6fef4964d9","Type":"ContainerStarted","Data":"c153355187a19dcabbdc8309658c69b00466297d3c30865eb93fb22cf989a64b"}
Jul 26 15:19:03.492952 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:03.492940    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:19:03.492996 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:03.492987    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:19:03.507689 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:03.507664    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:19:03.534777 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:03.534730    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx" podStartSLOduration=6.505428136 podStartE2EDuration="26.534717467s" totalImagesPullingTime="20.029289331s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:18:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:19:03.534145868 +0000 UTC m=+27.823977982" watchObservedRunningTime="2026-07-26 15:19:03.534717467 +0000 UTC m=+27.824549584"
Jul 26 15:19:03.555746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:03.555717    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:03.953910 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:03.953824    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:04.157812 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:04.157779    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:04.188282 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:04.188251    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:04.188473 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:04.188379    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:19:04.188473 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:04.188434    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:04.188584 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:04.188526    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:19:04.357926 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:04.357827    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:04.495113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:04.494950    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:19:04.513754 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:04.513718    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:19:04.747939 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:04.747840    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:04.800686 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:04.800657    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:05.188332 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:05.188250    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:05.188476 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:05.188347    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:19:05.343631 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:05.343598    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:05.499149 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:05.499042    2559 generic.go:356] "Generic (PLEG): container finished" podID="c95dc61e-9fe1-40b4-85cf-edb31a0c5607" containerID="8ae8b9515b74e20cc69e5d7da96ec271cfe144ca63ddf1011cb925ede40d7153" exitCode=0
Jul 26 15:19:05.499149 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:05.499130    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnzvq" event={"ID":"c95dc61e-9fe1-40b4-85cf-edb31a0c5607","Type":"ContainerDied","Data":"8ae8b9515b74e20cc69e5d7da96ec271cfe144ca63ddf1011cb925ede40d7153"}
Jul 26 15:19:05.545937 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:05.545906    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:05.737671 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:05.737639    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:05.945813 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:05.945731    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:06.085373 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:06.085340    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7wjc9"
Jul 26 15:19:06.085568 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:06.085484    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 26 15:19:06.086023 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:06.086002    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7wjc9"
Jul 26 15:19:06.149411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:06.149384    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:06.190562 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:06.190528    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:06.190562 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:06.190560    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:06.190747 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:06.190638    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:19:06.190785 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:06.190743    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:19:06.572716 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:06.572679    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:06.673522 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:06.673480    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:06.774148 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:06.774073    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:07.187665 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:07.187571    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:07.188250 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:07.187697    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:19:07.346166 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:07.346134    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:07.504929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:07.504843    2559 generic.go:356] "Generic (PLEG): container finished" podID="c95dc61e-9fe1-40b4-85cf-edb31a0c5607" containerID="fae0fb35cfccc92772af3c3f4d1b7ab9ba3007834e562a485fac70d3c13a6e38" exitCode=0
Jul 26 15:19:07.504929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:07.504893    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnzvq" event={"ID":"c95dc61e-9fe1-40b4-85cf-edb31a0c5607","Type":"ContainerDied","Data":"fae0fb35cfccc92772af3c3f4d1b7ab9ba3007834e562a485fac70d3c13a6e38"}
Jul 26 15:19:07.945698 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:07.945619    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:08.150774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:08.150741    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:08.187524 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:08.187481    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:08.187703 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:08.187607    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:19:08.187703 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:08.187648    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:08.188106 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:08.187724    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:19:08.200104 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:08.200029    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:08.349007 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:08.348967    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:08.545144 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:08.545058    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:08.849541 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:08.849445    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs\") pod \"network-metrics-daemon-p8dd2\" (UID: \"63512a39-e727-488a-96ea-e93a866aad7d\") " pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:08.849712 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:08.849646    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:19:08.849771 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:08.849725    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs podName:63512a39-e727-488a-96ea-e93a866aad7d nodeName:}" failed. No retries permitted until 2026-07-26 15:19:40.84970246 +0000 UTC m=+65.139534573 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs") pod "network-metrics-daemon-p8dd2" (UID: "63512a39-e727-488a-96ea-e93a866aad7d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:19:08.950307 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:08.950273    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret\") pod \"global-pull-secret-syncer-6g6lj\" (UID: \"5645fd38-99a5-4fa7-bf50-bf044c0444b6\") " pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:08.950506 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:08.950445    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:19:08.950564 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:08.950519    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret podName:5645fd38-99a5-4fa7-bf50-bf044c0444b6 nodeName:}" failed. No retries permitted until 2026-07-26 15:19:40.950499272 +0000 UTC m=+65.240331385 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret") pod "global-pull-secret-syncer-6g6lj" (UID: "5645fd38-99a5-4fa7-bf50-bf044c0444b6") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:19:09.051262 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:09.051221    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q8vl\" (UniqueName: \"kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl\") pod \"network-check-target-gztc2\" (UID: \"b003fa80-7770-4f06-9917-926f86feebf4\") " pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:09.051487 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:09.051421    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:19:09.051487 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:09.051448    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:19:09.051487 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:09.051464    2559 projected.go:196] Error preparing data for projected volume kube-api-access-5q8vl for pod openshift-network-diagnostics/network-check-target-gztc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:19:09.051657 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:09.051521    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl podName:b003fa80-7770-4f06-9917-926f86feebf4 nodeName:}" failed. No retries permitted until 2026-07-26 15:19:41.05150598 +0000 UTC m=+65.341338079 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5q8vl" (UniqueName: "kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl") pod "network-check-target-gztc2" (UID: "b003fa80-7770-4f06-9917-926f86feebf4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:19:09.187839 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:09.187758    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:09.188276 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:09.187893    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:19:09.547108 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:09.547015    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:09.678255 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:09.678220    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:09.779127 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:09.779091    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:09.879870 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:09.879781    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:10.150002 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:10.149918    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:10.187452 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:10.187418    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:10.187613 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:10.187418    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:10.187613 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:10.187567    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:19:10.187746 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:10.187650    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:19:10.348118 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:10.348081    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:10.750748 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:10.750698    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:11.187378 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:11.187286    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:11.187559 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:11.187433    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:19:11.345483 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:11.345446    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:11.517278 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:11.517248    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnzvq" event={"ID":"c95dc61e-9fe1-40b4-85cf-edb31a0c5607","Type":"ContainerStarted","Data":"cbd7b5f57af5f16d4a37f2d42afdea5c88eac7025a8f162dccca312329a11700"}
Jul 26 15:19:11.742099 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:11.742007    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:12.143910 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:12.143828    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:12.187309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:12.187279    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:12.187476 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:12.187448    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:19:12.187559 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:12.187491    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:12.188003 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:12.187918    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:19:12.350324 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:12.350290    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:12.521904 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:12.521827    2559 generic.go:356] "Generic (PLEG): container finished" podID="c95dc61e-9fe1-40b4-85cf-edb31a0c5607" containerID="cbd7b5f57af5f16d4a37f2d42afdea5c88eac7025a8f162dccca312329a11700" exitCode=0
Jul 26 15:19:12.521904 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:12.521864    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnzvq" event={"ID":"c95dc61e-9fe1-40b4-85cf-edb31a0c5607","Type":"ContainerDied","Data":"cbd7b5f57af5f16d4a37f2d42afdea5c88eac7025a8f162dccca312329a11700"}
Jul 26 15:19:12.550308 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:12.550272    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:13.187430 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:13.187389    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:13.187600 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:13.187497    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:19:13.526332 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:13.526246    2559 generic.go:356] "Generic (PLEG): container finished" podID="c95dc61e-9fe1-40b4-85cf-edb31a0c5607" containerID="5cf25781f64b849ec7c732e647a7da168844b9b6eeb70c60164710f4720f5eaf" exitCode=0
Jul 26 15:19:13.526332 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:13.526303    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnzvq" event={"ID":"c95dc61e-9fe1-40b4-85cf-edb31a0c5607","Type":"ContainerDied","Data":"5cf25781f64b849ec7c732e647a7da168844b9b6eeb70c60164710f4720f5eaf"}
Jul 26 15:19:14.187806 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:14.187608    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:14.187948 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:14.187608    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:14.187948 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:14.187881    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:19:14.188041 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:14.187948    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:19:14.531639 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:14.531552    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnzvq" event={"ID":"c95dc61e-9fe1-40b4-85cf-edb31a0c5607","Type":"ContainerStarted","Data":"110f367e802162e79f2f79971fc693a3e3e700dff1d5391766dc37755c23a8b9"}
Jul 26 15:19:14.552197 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:14.552147    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wnzvq" podStartSLOduration=7.464399634 podStartE2EDuration="37.552130721s" totalImagesPullingTime="30.087731087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:18:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:19:14.55057886 +0000 UTC m=+38.840410979" watchObservedRunningTime="2026-07-26 15:19:14.552130721 +0000 UTC m=+38.841962866"
Jul 26 15:19:15.187317 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:15.187280    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:15.187524 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:15.187403    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:19:16.188504 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:16.188465    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:16.188973 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:16.188593    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:19:16.188973 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:16.188643    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:16.188973 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:16.188768    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:19:17.188266 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:17.188231    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:17.188417 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:17.188336    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:19:17.388218 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:17.388180    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:18.187730 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:18.187699    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:18.187906 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:18.187704    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:18.187906 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:18.187806    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:19:18.187906 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:18.187872    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:19:19.187257 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:19.187218    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:19.187635 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:19.187322    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:19:19.425466 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:19.425434    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:19.551567 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:19.551483    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:19.647607 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:19.647580    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:19.749784 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:19.749752    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:19.843069 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:19.843008    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:20.187885 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:20.187808    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:20.188312 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:20.187938    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:19:20.188312 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:20.187987    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:20.188312 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:20.188096    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:19:20.811916 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:20.811882    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6g6lj"]
Jul 26 15:19:20.812168 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:20.812142    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:20.812322 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:20.812271    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:19:20.814707 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:20.814683    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p8dd2"]
Jul 26 15:19:20.814955 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:20.814935    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:20.815084 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:20.815062    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:19:20.825897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:20.825816    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gztc2"]
Jul 26 15:19:20.826028 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:20.825926    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:20.826028 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:20.826006    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:19:22.187951 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:22.187917    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:22.188338 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:22.187928    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:22.188338 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:22.188033    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:22.188338 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:22.188036    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:19:22.188338 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:22.188112    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:19:22.188338 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:22.188192    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:19:23.927683 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:23.927646    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:24.027534 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:24.027503    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:24.128287 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:24.128214    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:24.187734 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:24.187651    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:24.187901 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:24.187651    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:24.187901 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:24.187784    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8dd2" podUID="63512a39-e727-488a-96ea-e93a866aad7d"
Jul 26 15:19:24.187901 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:24.187651    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:24.187901 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:24.187826    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g6lj" podUID="5645fd38-99a5-4fa7-bf50-bf044c0444b6"
Jul 26 15:19:24.188106 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:24.187896    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gztc2" podUID="b003fa80-7770-4f06-9917-926f86feebf4"
Jul 26 15:19:24.270735 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:24.270673    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:24.342533 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:24.342467    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:24.414480 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:24.414446    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:24.617095 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:24.617063    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:25.492259 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.492062    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-110.ec2.internal" event="NodeReady"
Jul 26 15:19:25.492721 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.492380    2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 26 15:19:25.550688 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.550655    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-drfng"]
Jul 26 15:19:25.572400 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.572368    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5t4qj"]
Jul 26 15:19:25.572541 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.572478    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-drfng"
Jul 26 15:19:25.574532 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.574509    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 26 15:19:25.574657 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.574560    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4hft8\""
Jul 26 15:19:25.574657 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.574509    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 26 15:19:25.588304 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.588281    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5t4qj"]
Jul 26 15:19:25.588304 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.588311    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-drfng"]
Jul 26 15:19:25.588448 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.588390    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5t4qj"
Jul 26 15:19:25.590252 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.590229    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 26 15:19:25.590356 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.590256    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-s7dmz\""
Jul 26 15:19:25.590356 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.590264    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 26 15:19:25.590356 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.590261    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 26 15:19:25.650254 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.650216    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-s7hbj"]
Jul 26 15:19:25.664980 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.664953    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:25.670616 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.670594    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s7hbj"]
Jul 26 15:19:25.670734 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.670713    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s7hbj"
Jul 26 15:19:25.672782 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.672761    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 26 15:19:25.672782 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.672777    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-flhls\""
Jul 26 15:19:25.672940 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.672761    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 26 15:19:25.672940 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.672761    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 26 15:19:25.672940 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.672815    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 26 15:19:25.676560 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.676534    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1a024411-b6f8-46e8-a695-84c440761063-config-volume\") pod \"dns-default-drfng\" (UID: \"1a024411-b6f8-46e8-a695-84c440761063\") " pod="openshift-dns/dns-default-drfng"
Jul 26 15:19:25.676648 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.676632    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a024411-b6f8-46e8-a695-84c440761063-tmp-dir\") pod \"dns-default-drfng\" (UID: \"1a024411-b6f8-46e8-a695-84c440761063\") " pod="openshift-dns/dns-default-drfng"
Jul 26 15:19:25.676684 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.676676    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v826c\" (UniqueName: \"kubernetes.io/projected/1a024411-b6f8-46e8-a695-84c440761063-kube-api-access-v826c\") pod \"dns-default-drfng\" (UID: \"1a024411-b6f8-46e8-a695-84c440761063\") " pod="openshift-dns/dns-default-drfng"
Jul 26 15:19:25.676750 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.676735    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a024411-b6f8-46e8-a695-84c440761063-metrics-tls\") pod \"dns-default-drfng\" (UID: \"1a024411-b6f8-46e8-a695-84c440761063\") " pod="openshift-dns/dns-default-drfng"
Jul 26 15:19:25.777748 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.777662    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d91f8c8-c702-4f74-a29e-af54043a79d5-data-volume\") pod \"insights-runtime-extractor-s7hbj\" (UID: \"7d91f8c8-c702-4f74-a29e-af54043a79d5\") " pod="openshift-insights/insights-runtime-extractor-s7hbj"
Jul 26 15:19:25.777748 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.777697    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d91f8c8-c702-4f74-a29e-af54043a79d5-crio-socket\") pod \"insights-runtime-extractor-s7hbj\" (UID: \"7d91f8c8-c702-4f74-a29e-af54043a79d5\") " pod="openshift-insights/insights-runtime-extractor-s7hbj"
Jul 26 15:19:25.777748 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.777730    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d91f8c8-c702-4f74-a29e-af54043a79d5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s7hbj\" (UID: \"7d91f8c8-c702-4f74-a29e-af54043a79d5\") " pod="openshift-insights/insights-runtime-extractor-s7hbj"
Jul 26 15:19:25.778026 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.777794    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1a024411-b6f8-46e8-a695-84c440761063-config-volume\") pod \"dns-default-drfng\" (UID: \"1a024411-b6f8-46e8-a695-84c440761063\") " pod="openshift-dns/dns-default-drfng"
Jul 26 15:19:25.778026 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.777842    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sktx\" (UniqueName: \"kubernetes.io/projected/7d91f8c8-c702-4f74-a29e-af54043a79d5-kube-api-access-5sktx\") pod \"insights-runtime-extractor-s7hbj\" (UID: \"7d91f8c8-c702-4f74-a29e-af54043a79d5\") " pod="openshift-insights/insights-runtime-extractor-s7hbj"
Jul 26 15:19:25.778026 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.777912    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a024411-b6f8-46e8-a695-84c440761063-tmp-dir\") pod \"dns-default-drfng\" (UID: \"1a024411-b6f8-46e8-a695-84c440761063\") " pod="openshift-dns/dns-default-drfng"
Jul 26 15:19:25.778026 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.777934    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/487000fe-719e-4b3a-be94-d9d0cbcd94db-cert\") pod \"ingress-canary-5t4qj\" (UID: \"487000fe-719e-4b3a-be94-d9d0cbcd94db\") " pod="openshift-ingress-canary/ingress-canary-5t4qj"
Jul 26 15:19:25.778026 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.777974    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7qr5\" (UniqueName: \"kubernetes.io/projected/487000fe-719e-4b3a-be94-d9d0cbcd94db-kube-api-access-w7qr5\") pod \"ingress-canary-5t4qj\" (UID: \"487000fe-719e-4b3a-be94-d9d0cbcd94db\") " pod="openshift-ingress-canary/ingress-canary-5t4qj"
Jul 26 15:19:25.778026 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.778002    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v826c\" (UniqueName: \"kubernetes.io/projected/1a024411-b6f8-46e8-a695-84c440761063-kube-api-access-v826c\") pod \"dns-default-drfng\" (UID: \"1a024411-b6f8-46e8-a695-84c440761063\") " pod="openshift-dns/dns-default-drfng"
Jul 26 15:19:25.778282 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.778048    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d91f8c8-c702-4f74-a29e-af54043a79d5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s7hbj\" (UID: \"7d91f8c8-c702-4f74-a29e-af54043a79d5\") " pod="openshift-insights/insights-runtime-extractor-s7hbj"
Jul 26 15:19:25.778282 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.778077    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a024411-b6f8-46e8-a695-84c440761063-metrics-tls\") pod \"dns-default-drfng\" (UID: \"1a024411-b6f8-46e8-a695-84c440761063\") " pod="openshift-dns/dns-default-drfng"
Jul 26 15:19:25.778282 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.778237    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a024411-b6f8-46e8-a695-84c440761063-tmp-dir\") pod \"dns-default-drfng\" (UID: \"1a024411-b6f8-46e8-a695-84c440761063\") " pod="openshift-dns/dns-default-drfng"
Jul 26 15:19:25.778415 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.778393    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1a024411-b6f8-46e8-a695-84c440761063-config-volume\") pod \"dns-default-drfng\" (UID: \"1a024411-b6f8-46e8-a695-84c440761063\") " pod="openshift-dns/dns-default-drfng"
Jul 26 15:19:25.782171 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.782153    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a024411-b6f8-46e8-a695-84c440761063-metrics-tls\") pod \"dns-default-drfng\" (UID: \"1a024411-b6f8-46e8-a695-84c440761063\") " pod="openshift-dns/dns-default-drfng"
Jul 26 15:19:25.785601 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.785578    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v826c\" (UniqueName: \"kubernetes.io/projected/1a024411-b6f8-46e8-a695-84c440761063-kube-api-access-v826c\") pod \"dns-default-drfng\" (UID: \"1a024411-b6f8-46e8-a695-84c440761063\") " pod="openshift-dns/dns-default-drfng"
Jul 26 15:19:25.878685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.878645    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d91f8c8-c702-4f74-a29e-af54043a79d5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s7hbj\" (UID: \"7d91f8c8-c702-4f74-a29e-af54043a79d5\") " pod="openshift-insights/insights-runtime-extractor-s7hbj"
Jul 26 15:19:25.878685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.878685    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d91f8c8-c702-4f74-a29e-af54043a79d5-data-volume\") pod \"insights-runtime-extractor-s7hbj\" (UID: \"7d91f8c8-c702-4f74-a29e-af54043a79d5\") " pod="openshift-insights/insights-runtime-extractor-s7hbj"
Jul 26 15:19:25.878918 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.878705    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d91f8c8-c702-4f74-a29e-af54043a79d5-crio-socket\") pod \"insights-runtime-extractor-s7hbj\" (UID: \"7d91f8c8-c702-4f74-a29e-af54043a79d5\") " pod="openshift-insights/insights-runtime-extractor-s7hbj"
Jul 26 15:19:25.878918 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.878725    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d91f8c8-c702-4f74-a29e-af54043a79d5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s7hbj\" (UID: \"7d91f8c8-c702-4f74-a29e-af54043a79d5\") " pod="openshift-insights/insights-runtime-extractor-s7hbj"
Jul 26 15:19:25.878918 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.878861    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sktx\" (UniqueName: \"kubernetes.io/projected/7d91f8c8-c702-4f74-a29e-af54043a79d5-kube-api-access-5sktx\") pod \"insights-runtime-extractor-s7hbj\" (UID: \"7d91f8c8-c702-4f74-a29e-af54043a79d5\") " pod="openshift-insights/insights-runtime-extractor-s7hbj"
Jul 26 15:19:25.879070 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.878942    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/487000fe-719e-4b3a-be94-d9d0cbcd94db-cert\") pod \"ingress-canary-5t4qj\" (UID: \"487000fe-719e-4b3a-be94-d9d0cbcd94db\") " pod="openshift-ingress-canary/ingress-canary-5t4qj"
Jul 26 15:19:25.879070 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.878965    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d91f8c8-c702-4f74-a29e-af54043a79d5-crio-socket\") pod \"insights-runtime-extractor-s7hbj\" (UID: \"7d91f8c8-c702-4f74-a29e-af54043a79d5\") " pod="openshift-insights/insights-runtime-extractor-s7hbj"
Jul 26 15:19:25.879070 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.878984    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7qr5\" (UniqueName: \"kubernetes.io/projected/487000fe-719e-4b3a-be94-d9d0cbcd94db-kube-api-access-w7qr5\") pod \"ingress-canary-5t4qj\" (UID: \"487000fe-719e-4b3a-be94-d9d0cbcd94db\") " pod="openshift-ingress-canary/ingress-canary-5t4qj"
Jul 26 15:19:25.879070 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.879063    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d91f8c8-c702-4f74-a29e-af54043a79d5-data-volume\") pod \"insights-runtime-extractor-s7hbj\" (UID: \"7d91f8c8-c702-4f74-a29e-af54043a79d5\") " pod="openshift-insights/insights-runtime-extractor-s7hbj"
Jul 26 15:19:25.879404 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.879379    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d91f8c8-c702-4f74-a29e-af54043a79d5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s7hbj\" (UID: \"7d91f8c8-c702-4f74-a29e-af54043a79d5\") " pod="openshift-insights/insights-runtime-extractor-s7hbj"
Jul 26 15:19:25.881155 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.881129    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d91f8c8-c702-4f74-a29e-af54043a79d5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s7hbj\" (UID: \"7d91f8c8-c702-4f74-a29e-af54043a79d5\") " pod="openshift-insights/insights-runtime-extractor-s7hbj"
Jul 26 15:19:25.881284 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.881253    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/487000fe-719e-4b3a-be94-d9d0cbcd94db-cert\") pod \"ingress-canary-5t4qj\" (UID: \"487000fe-719e-4b3a-be94-d9d0cbcd94db\") " pod="openshift-ingress-canary/ingress-canary-5t4qj"
Jul 26 15:19:25.884420 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.884401    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-drfng"
Jul 26 15:19:25.886118 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.886086    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7qr5\" (UniqueName: \"kubernetes.io/projected/487000fe-719e-4b3a-be94-d9d0cbcd94db-kube-api-access-w7qr5\") pod \"ingress-canary-5t4qj\" (UID: \"487000fe-719e-4b3a-be94-d9d0cbcd94db\") " pod="openshift-ingress-canary/ingress-canary-5t4qj"
Jul 26 15:19:25.886635 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.886615    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sktx\" (UniqueName: \"kubernetes.io/projected/7d91f8c8-c702-4f74-a29e-af54043a79d5-kube-api-access-5sktx\") pod \"insights-runtime-extractor-s7hbj\" (UID: \"7d91f8c8-c702-4f74-a29e-af54043a79d5\") " pod="openshift-insights/insights-runtime-extractor-s7hbj"
Jul 26 15:19:25.896675 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.896651    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5t4qj"
Jul 26 15:19:25.979655 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:25.979624    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s7hbj"
Jul 26 15:19:26.102606 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:26.102577    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-drfng"]
Jul 26 15:19:26.116064 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:26.116033    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5t4qj"]
Jul 26 15:19:26.119701 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:19:26.119679    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod487000fe_719e_4b3a_be94_d9d0cbcd94db.slice/crio-5a2f5d07d3b0f8bc9d2c7d9a9c8ec82923c6fc49ef084992c8e8220cc0968a86 WatchSource:0}: Error finding container 5a2f5d07d3b0f8bc9d2c7d9a9c8ec82923c6fc49ef084992c8e8220cc0968a86: Status 404 returned error can't find the container with id 5a2f5d07d3b0f8bc9d2c7d9a9c8ec82923c6fc49ef084992c8e8220cc0968a86
Jul 26 15:19:26.148547 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:26.148518    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s7hbj"]
Jul 26 15:19:26.151704 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:19:26.151681    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d91f8c8_c702_4f74_a29e_af54043a79d5.slice/crio-24d9baedeeee26c16c4ab300b843cd9f2da4cba12a7a8674724ea86ba54a919c WatchSource:0}: Error finding container 24d9baedeeee26c16c4ab300b843cd9f2da4cba12a7a8674724ea86ba54a919c: Status 404 returned error can't find the container with id 24d9baedeeee26c16c4ab300b843cd9f2da4cba12a7a8674724ea86ba54a919c
Jul 26 15:19:26.189221 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:26.189184    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:26.189396 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:26.189302    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:26.189396 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:26.189372    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:26.191428 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:26.191405    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 26 15:19:26.191561 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:26.191405    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 26 15:19:26.191561 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:26.191410    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 26 15:19:26.191804 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:26.191783    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 26 15:19:26.191906 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:26.191785    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bcjwm\""
Jul 26 15:19:26.191906 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:26.191818    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6xwws\""
Jul 26 15:19:26.557418 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:26.557377    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s7hbj" event={"ID":"7d91f8c8-c702-4f74-a29e-af54043a79d5","Type":"ContainerStarted","Data":"24d9baedeeee26c16c4ab300b843cd9f2da4cba12a7a8674724ea86ba54a919c"}
Jul 26 15:19:26.558311 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:26.558285    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5t4qj" event={"ID":"487000fe-719e-4b3a-be94-d9d0cbcd94db","Type":"ContainerStarted","Data":"5a2f5d07d3b0f8bc9d2c7d9a9c8ec82923c6fc49ef084992c8e8220cc0968a86"}
Jul 26 15:19:26.559189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:26.559166    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-drfng" event={"ID":"1a024411-b6f8-46e8-a695-84c440761063","Type":"ContainerStarted","Data":"75c0cad7c695f87f093ce70c8dce0e723bb366049d0e260ed1b774fdf68e83e8"}
Jul 26 15:19:30.286726 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.286459    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:30.325214 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.325163    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-hr8zw"]
Jul 26 15:19:30.387889 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.387855    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:30.478596 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.478567    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-hr8zw"]
Jul 26 15:19:30.478777 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.478694    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-hr8zw"
Jul 26 15:19:30.480874 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.480849    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 26 15:19:30.480874 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.480847    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 26 15:19:30.481457 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.481365    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 26 15:19:30.481653 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.481460    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 26 15:19:30.481653 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.481621    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-nndrs\""
Jul 26 15:19:30.481765 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.481751    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 26 15:19:30.487710 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.487552    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:30.515910 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.515869    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d18e961-8a3e-469e-8357-cd70054a719d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-hr8zw\" (UID: \"8d18e961-8a3e-469e-8357-cd70054a719d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hr8zw"
Jul 26 15:19:30.516080 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.515927    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8d18e961-8a3e-469e-8357-cd70054a719d-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-hr8zw\" (UID: \"8d18e961-8a3e-469e-8357-cd70054a719d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hr8zw"
Jul 26 15:19:30.516080 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.515956    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d18e961-8a3e-469e-8357-cd70054a719d-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-hr8zw\" (UID: \"8d18e961-8a3e-469e-8357-cd70054a719d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hr8zw"
Jul 26 15:19:30.516080 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.516047    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvm56\" (UniqueName: \"kubernetes.io/projected/8d18e961-8a3e-469e-8357-cd70054a719d-kube-api-access-gvm56\") pod \"prometheus-operator-84d69f794d-hr8zw\" (UID: \"8d18e961-8a3e-469e-8357-cd70054a719d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hr8zw"
Jul 26 15:19:30.616758 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.616723    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvm56\" (UniqueName: \"kubernetes.io/projected/8d18e961-8a3e-469e-8357-cd70054a719d-kube-api-access-gvm56\") pod \"prometheus-operator-84d69f794d-hr8zw\" (UID: \"8d18e961-8a3e-469e-8357-cd70054a719d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hr8zw"
Jul 26 15:19:30.616970 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.616783    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d18e961-8a3e-469e-8357-cd70054a719d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-hr8zw\" (UID: \"8d18e961-8a3e-469e-8357-cd70054a719d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hr8zw"
Jul 26 15:19:30.616970 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.616947    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8d18e961-8a3e-469e-8357-cd70054a719d-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-hr8zw\" (UID: \"8d18e961-8a3e-469e-8357-cd70054a719d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hr8zw"
Jul 26 15:19:30.617086 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.616996    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d18e961-8a3e-469e-8357-cd70054a719d-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-hr8zw\" (UID: \"8d18e961-8a3e-469e-8357-cd70054a719d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hr8zw"
Jul 26 15:19:30.617766 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.617733    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d18e961-8a3e-469e-8357-cd70054a719d-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-hr8zw\" (UID: \"8d18e961-8a3e-469e-8357-cd70054a719d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hr8zw"
Jul 26 15:19:30.620256 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.620232    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d18e961-8a3e-469e-8357-cd70054a719d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-hr8zw\" (UID: \"8d18e961-8a3e-469e-8357-cd70054a719d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hr8zw"
Jul 26 15:19:30.620386 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.620370    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8d18e961-8a3e-469e-8357-cd70054a719d-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-hr8zw\" (UID: \"8d18e961-8a3e-469e-8357-cd70054a719d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hr8zw"
Jul 26 15:19:30.624263 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.624216    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvm56\" (UniqueName: \"kubernetes.io/projected/8d18e961-8a3e-469e-8357-cd70054a719d-kube-api-access-gvm56\") pod \"prometheus-operator-84d69f794d-hr8zw\" (UID: \"8d18e961-8a3e-469e-8357-cd70054a719d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hr8zw"
Jul 26 15:19:30.790580 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:30.790497    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-hr8zw"
Jul 26 15:19:31.271279 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:31.271247    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:31.325570 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:31.325539    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:32.625313 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:32.624942    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-hr8zw"]
Jul 26 15:19:32.633594 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:19:32.633543    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d18e961_8a3e_469e_8357_cd70054a719d.slice/crio-fdeb4a1c8224550b90b0e3993b051e10b80dda78414d9aeecf33a939d382d359 WatchSource:0}: Error finding container fdeb4a1c8224550b90b0e3993b051e10b80dda78414d9aeecf33a939d382d359: Status 404 returned error can't find the container with id fdeb4a1c8224550b90b0e3993b051e10b80dda78414d9aeecf33a939d382d359
Jul 26 15:19:33.581354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:33.581305    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-drfng" event={"ID":"1a024411-b6f8-46e8-a695-84c440761063","Type":"ContainerStarted","Data":"9135aa37fe4de57c3e2e33a55a2c99ecccae0299009bcebe542d0a8934a3f789"}
Jul 26 15:19:33.581550 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:33.581378    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-drfng" event={"ID":"1a024411-b6f8-46e8-a695-84c440761063","Type":"ContainerStarted","Data":"7836eb54c95ab382156d44246ff07f4dd593a7d8772d567ff6b778ac0c05b791"}
Jul 26 15:19:33.581550 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:33.581423    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-drfng"
Jul 26 15:19:33.583054 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:33.583005    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s7hbj" event={"ID":"7d91f8c8-c702-4f74-a29e-af54043a79d5","Type":"ContainerStarted","Data":"ecae6ac9163d113d39168e8ffe857d959c372b871c0283362c40b600fff07f04"}
Jul 26 15:19:33.584755 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:33.584708    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5t4qj" event={"ID":"487000fe-719e-4b3a-be94-d9d0cbcd94db","Type":"ContainerStarted","Data":"8452e9b361d27721d031ae19d8eaee88eb326fe0bc5b5b1332caa95d2ea03569"}
Jul 26 15:19:33.586159 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:33.586138    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-hr8zw" event={"ID":"8d18e961-8a3e-469e-8357-cd70054a719d","Type":"ContainerStarted","Data":"fdeb4a1c8224550b90b0e3993b051e10b80dda78414d9aeecf33a939d382d359"}
Jul 26 15:19:33.597047 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:33.596993    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-drfng" podStartSLOduration=2.212920798 podStartE2EDuration="8.596975667s" totalImagesPullingTime="6.384054869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:19:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:19:33.595606951 +0000 UTC m=+57.885439074" watchObservedRunningTime="2026-07-26 15:19:33.596975667 +0000 UTC m=+57.886807786"
Jul 26 15:19:33.610052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:33.609729    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5t4qj" podStartSLOduration=2.236819152 podStartE2EDuration="8.609707795s" totalImagesPullingTime="6.372888643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:19:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:19:33.609080686 +0000 UTC m=+57.898912806" watchObservedRunningTime="2026-07-26 15:19:33.609707795 +0000 UTC m=+57.899539915"
Jul 26 15:19:33.991662 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:33.991636    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:34.092645 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:34.092602    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:34.203233 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:34.203181    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:34.294675 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:34.294591    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:34.592024 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:34.591988    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-hr8zw" event={"ID":"8d18e961-8a3e-469e-8357-cd70054a719d","Type":"ContainerStarted","Data":"4b218bb877bb45c06a4c3cd94c8f80c19bfa7aa34f9bf9d4ad6b25f60df11bbe"}
Jul 26 15:19:34.592195 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:34.592052    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-hr8zw" event={"ID":"8d18e961-8a3e-469e-8357-cd70054a719d","Type":"ContainerStarted","Data":"94ba46e572c6fb857b1aebfe5fa4febfe2e3435b90299cbcacadaa083951918c"}
Jul 26 15:19:34.607761 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:34.607532    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-hr8zw" podStartSLOduration=3.284956131 podStartE2EDuration="4.607510608s" totalImagesPullingTime="1.322554477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:19:34.607137011 +0000 UTC m=+58.896969132" watchObservedRunningTime="2026-07-26 15:19:34.607510608 +0000 UTC m=+58.897342727"
Jul 26 15:19:36.431658 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.431616    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:36.527493 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.527464    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qxgsx"
Jul 26 15:19:36.648113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.648064    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:36.687177 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.687105    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp"]
Jul 26 15:19:36.690047 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.690027    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp"
Jul 26 15:19:36.692129 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.692106    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-hkncv\""
Jul 26 15:19:36.692355 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.692334    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 26 15:19:36.692413 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.692339    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 26 15:19:36.701176 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.701147    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp"]
Jul 26 15:19:36.704718 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.704697    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"]
Jul 26 15:19:36.706581 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.706564    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.708479 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.708458    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 26 15:19:36.708595 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.708462    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 26 15:19:36.708712 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.708689    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-p6ghp\""
Jul 26 15:19:36.708889 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.708860    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 26 15:19:36.724188 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.724158    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"]
Jul 26 15:19:36.726437 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.726413    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rlzvj"]
Jul 26 15:19:36.728572 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.728556    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.730479 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.730457    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 26 15:19:36.730583 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.730557    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7sx6d\""
Jul 26 15:19:36.730649 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.730557    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 26 15:19:36.730718 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.730637    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 26 15:19:36.734759 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.734657    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:36.767655 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.767622    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cab5fcb1-28bd-4602-937e-34b17c35871b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.767655 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.767659    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cab5fcb1-28bd-4602-937e-34b17c35871b-sys\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.767920 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.767678    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cab5fcb1-28bd-4602-937e-34b17c35871b-metrics-client-ca\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.767920 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.767696    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4a05eea-40b1-4d16-b333-72d815bc2b6b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-cjjgp\" (UID: \"c4a05eea-40b1-4d16-b333-72d815bc2b6b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp"
Jul 26 15:19:36.767920 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.767778    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4a05eea-40b1-4d16-b333-72d815bc2b6b-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-cjjgp\" (UID: \"c4a05eea-40b1-4d16-b333-72d815bc2b6b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp"
Jul 26 15:19:36.767920 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.767815    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cab5fcb1-28bd-4602-937e-34b17c35871b-node-exporter-tls\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.767920 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.767847    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj66g\" (UniqueName: \"kubernetes.io/projected/c4a05eea-40b1-4d16-b333-72d815bc2b6b-kube-api-access-nj66g\") pod \"openshift-state-metrics-b8f49f5c6-cjjgp\" (UID: \"c4a05eea-40b1-4d16-b333-72d815bc2b6b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp"
Jul 26 15:19:36.767920 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.767873    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4a05eea-40b1-4d16-b333-72d815bc2b6b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-cjjgp\" (UID: \"c4a05eea-40b1-4d16-b333-72d815bc2b6b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp"
Jul 26 15:19:36.767920 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.767903    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cab5fcb1-28bd-4602-937e-34b17c35871b-root\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.768338 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.767948    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcc9m\" (UniqueName: \"kubernetes.io/projected/70542744-fd1a-4471-b33d-80747ea33774-kube-api-access-hcc9m\") pod \"kube-state-metrics-56d9f944b4-7h29h\" (UID: \"70542744-fd1a-4471-b33d-80747ea33774\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.768338 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.767969    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69qlz\" (UniqueName: \"kubernetes.io/projected/cab5fcb1-28bd-4602-937e-34b17c35871b-kube-api-access-69qlz\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.768338 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.767985    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cab5fcb1-28bd-4602-937e-34b17c35871b-node-exporter-accelerators-collector-config\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.768338 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.768006    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cab5fcb1-28bd-4602-937e-34b17c35871b-node-exporter-wtmp\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.768338 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.768022    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/70542744-fd1a-4471-b33d-80747ea33774-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-7h29h\" (UID: \"70542744-fd1a-4471-b33d-80747ea33774\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.768338 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.768087    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/70542744-fd1a-4471-b33d-80747ea33774-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-7h29h\" (UID: \"70542744-fd1a-4471-b33d-80747ea33774\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.768338 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.768123    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70542744-fd1a-4471-b33d-80747ea33774-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-7h29h\" (UID: \"70542744-fd1a-4471-b33d-80747ea33774\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.768338 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.768157    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70542744-fd1a-4471-b33d-80747ea33774-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-7h29h\" (UID: \"70542744-fd1a-4471-b33d-80747ea33774\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.768338 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.768188    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cab5fcb1-28bd-4602-937e-34b17c35871b-node-exporter-textfile\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.768338 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.768236    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/70542744-fd1a-4471-b33d-80747ea33774-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-7h29h\" (UID: \"70542744-fd1a-4471-b33d-80747ea33774\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.802608 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.802572    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:36.868880 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.868839    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj66g\" (UniqueName: \"kubernetes.io/projected/c4a05eea-40b1-4d16-b333-72d815bc2b6b-kube-api-access-nj66g\") pod \"openshift-state-metrics-b8f49f5c6-cjjgp\" (UID: \"c4a05eea-40b1-4d16-b333-72d815bc2b6b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp"
Jul 26 15:19:36.868880 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.868882    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4a05eea-40b1-4d16-b333-72d815bc2b6b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-cjjgp\" (UID: \"c4a05eea-40b1-4d16-b333-72d815bc2b6b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp"
Jul 26 15:19:36.869130 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.868913    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cab5fcb1-28bd-4602-937e-34b17c35871b-root\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.869130 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.868953    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcc9m\" (UniqueName: \"kubernetes.io/projected/70542744-fd1a-4471-b33d-80747ea33774-kube-api-access-hcc9m\") pod \"kube-state-metrics-56d9f944b4-7h29h\" (UID: \"70542744-fd1a-4471-b33d-80747ea33774\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.869130 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.868978    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69qlz\" (UniqueName: \"kubernetes.io/projected/cab5fcb1-28bd-4602-937e-34b17c35871b-kube-api-access-69qlz\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.869130 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:36.868997    2559 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 26 15:19:36.869130 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869004    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cab5fcb1-28bd-4602-937e-34b17c35871b-node-exporter-accelerators-collector-config\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.869130 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869035    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cab5fcb1-28bd-4602-937e-34b17c35871b-node-exporter-wtmp\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.869130 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869043    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cab5fcb1-28bd-4602-937e-34b17c35871b-root\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.869130 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:36.869069    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4a05eea-40b1-4d16-b333-72d815bc2b6b-openshift-state-metrics-tls podName:c4a05eea-40b1-4d16-b333-72d815bc2b6b nodeName:}" failed. No retries permitted until 2026-07-26 15:19:37.369047376 +0000 UTC m=+61.658879476 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c4a05eea-40b1-4d16-b333-72d815bc2b6b-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-cjjgp" (UID: "c4a05eea-40b1-4d16-b333-72d815bc2b6b") : secret "openshift-state-metrics-tls" not found
Jul 26 15:19:36.869130 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869094    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/70542744-fd1a-4471-b33d-80747ea33774-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-7h29h\" (UID: \"70542744-fd1a-4471-b33d-80747ea33774\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.869612 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869141    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/70542744-fd1a-4471-b33d-80747ea33774-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-7h29h\" (UID: \"70542744-fd1a-4471-b33d-80747ea33774\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.869612 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869177    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cab5fcb1-28bd-4602-937e-34b17c35871b-node-exporter-wtmp\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.869612 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869178    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70542744-fd1a-4471-b33d-80747ea33774-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-7h29h\" (UID: \"70542744-fd1a-4471-b33d-80747ea33774\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.869612 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869255    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70542744-fd1a-4471-b33d-80747ea33774-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-7h29h\" (UID: \"70542744-fd1a-4471-b33d-80747ea33774\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.869612 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869290    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cab5fcb1-28bd-4602-937e-34b17c35871b-node-exporter-textfile\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.869612 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869325    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/70542744-fd1a-4471-b33d-80747ea33774-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-7h29h\" (UID: \"70542744-fd1a-4471-b33d-80747ea33774\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.869612 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869357    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cab5fcb1-28bd-4602-937e-34b17c35871b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.869612 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869384    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cab5fcb1-28bd-4602-937e-34b17c35871b-sys\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.869612 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869420    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cab5fcb1-28bd-4602-937e-34b17c35871b-metrics-client-ca\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.869612 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869455    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4a05eea-40b1-4d16-b333-72d815bc2b6b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-cjjgp\" (UID: \"c4a05eea-40b1-4d16-b333-72d815bc2b6b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp"
Jul 26 15:19:36.869612 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869510    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4a05eea-40b1-4d16-b333-72d815bc2b6b-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-cjjgp\" (UID: \"c4a05eea-40b1-4d16-b333-72d815bc2b6b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp"
Jul 26 15:19:36.869612 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869539    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cab5fcb1-28bd-4602-937e-34b17c35871b-node-exporter-tls\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.870360 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:36.869656    2559 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 26 15:19:36.870360 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:36.869703    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cab5fcb1-28bd-4602-937e-34b17c35871b-node-exporter-tls podName:cab5fcb1-28bd-4602-937e-34b17c35871b nodeName:}" failed. No retries permitted until 2026-07-26 15:19:37.369688177 +0000 UTC m=+61.659520277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/cab5fcb1-28bd-4602-937e-34b17c35871b-node-exporter-tls") pod "node-exporter-rlzvj" (UID: "cab5fcb1-28bd-4602-937e-34b17c35871b") : secret "node-exporter-tls" not found
Jul 26 15:19:36.870360 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869724    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cab5fcb1-28bd-4602-937e-34b17c35871b-node-exporter-accelerators-collector-config\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.870360 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869731    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/70542744-fd1a-4471-b33d-80747ea33774-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-7h29h\" (UID: \"70542744-fd1a-4471-b33d-80747ea33774\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.870360 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869972    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cab5fcb1-28bd-4602-937e-34b17c35871b-node-exporter-textfile\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.870360 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.869986    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70542744-fd1a-4471-b33d-80747ea33774-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-7h29h\" (UID: \"70542744-fd1a-4471-b33d-80747ea33774\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.870360 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.870013    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/70542744-fd1a-4471-b33d-80747ea33774-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-7h29h\" (UID: \"70542744-fd1a-4471-b33d-80747ea33774\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.870360 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.870048    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cab5fcb1-28bd-4602-937e-34b17c35871b-sys\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.870824 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.870645    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cab5fcb1-28bd-4602-937e-34b17c35871b-metrics-client-ca\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.870824 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.870687    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4a05eea-40b1-4d16-b333-72d815bc2b6b-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-cjjgp\" (UID: \"c4a05eea-40b1-4d16-b333-72d815bc2b6b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp"
Jul 26 15:19:36.871844 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.871812    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70542744-fd1a-4471-b33d-80747ea33774-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-7h29h\" (UID: \"70542744-fd1a-4471-b33d-80747ea33774\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.871958 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.871904    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cab5fcb1-28bd-4602-937e-34b17c35871b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.871958 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.871934    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/70542744-fd1a-4471-b33d-80747ea33774-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-7h29h\" (UID: \"70542744-fd1a-4471-b33d-80747ea33774\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.872100 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.872082    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4a05eea-40b1-4d16-b333-72d815bc2b6b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-cjjgp\" (UID: \"c4a05eea-40b1-4d16-b333-72d815bc2b6b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp"
Jul 26 15:19:36.877387 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.877360    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj66g\" (UniqueName: \"kubernetes.io/projected/c4a05eea-40b1-4d16-b333-72d815bc2b6b-kube-api-access-nj66g\") pod \"openshift-state-metrics-b8f49f5c6-cjjgp\" (UID: \"c4a05eea-40b1-4d16-b333-72d815bc2b6b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp"
Jul 26 15:19:36.877572 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.877442    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69qlz\" (UniqueName: \"kubernetes.io/projected/cab5fcb1-28bd-4602-937e-34b17c35871b-kube-api-access-69qlz\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:36.877662 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.877647    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcc9m\" (UniqueName: \"kubernetes.io/projected/70542744-fd1a-4471-b33d-80747ea33774-kube-api-access-hcc9m\") pod \"kube-state-metrics-56d9f944b4-7h29h\" (UID: \"70542744-fd1a-4471-b33d-80747ea33774\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:36.904226 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:36.904173    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:37.007993 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.007910    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:37.016055 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.016027    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"
Jul 26 15:19:37.104538 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.104415    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:37.166123 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.166090    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h"]
Jul 26 15:19:37.169324 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:19:37.169291    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70542744_fd1a_4471_b33d_80747ea33774.slice/crio-0f1f9f66a7aa0f0aa0335e792fcbbe3c91bc2202729b1c3806df4eb34e0f5bb1 WatchSource:0}: Error finding container 0f1f9f66a7aa0f0aa0335e792fcbbe3c91bc2202729b1c3806df4eb34e0f5bb1: Status 404 returned error can't find the container with id 0f1f9f66a7aa0f0aa0335e792fcbbe3c91bc2202729b1c3806df4eb34e0f5bb1
Jul 26 15:19:37.255405 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.255363    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:37.373219 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.373173    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cab5fcb1-28bd-4602-937e-34b17c35871b-node-exporter-tls\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:37.373414 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.373249    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4a05eea-40b1-4d16-b333-72d815bc2b6b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-cjjgp\" (UID: \"c4a05eea-40b1-4d16-b333-72d815bc2b6b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp"
Jul 26 15:19:37.375940 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.375911    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cab5fcb1-28bd-4602-937e-34b17c35871b-node-exporter-tls\") pod \"node-exporter-rlzvj\" (UID: \"cab5fcb1-28bd-4602-937e-34b17c35871b\") " pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:37.376054 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.375965    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4a05eea-40b1-4d16-b333-72d815bc2b6b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-cjjgp\" (UID: \"c4a05eea-40b1-4d16-b333-72d815bc2b6b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp"
Jul 26 15:19:37.598898 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.598859    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp"
Jul 26 15:19:37.603940 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.603893    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h" event={"ID":"70542744-fd1a-4471-b33d-80747ea33774","Type":"ContainerStarted","Data":"0f1f9f66a7aa0f0aa0335e792fcbbe3c91bc2202729b1c3806df4eb34e0f5bb1"}
Jul 26 15:19:37.606639 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.606606    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s7hbj" event={"ID":"7d91f8c8-c702-4f74-a29e-af54043a79d5","Type":"ContainerStarted","Data":"5470d3641cc71ac7e5db9a78a848e6f9f3dbfc304cc45b9ea5d9f83b8bbca09e"}
Jul 26 15:19:37.637060 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.636941    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rlzvj"
Jul 26 15:19:37.670321 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.670259    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:37.760323 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.760288    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:37.766462 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.765107    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 26 15:19:37.773466 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.773391    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp"]
Jul 26 15:19:37.773607 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.773572    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.777454 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.775938    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 26 15:19:37.777454 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.776584    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 26 15:19:37.777454 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.776608    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 26 15:19:37.777454 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.776781    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 26 15:19:37.777454 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.776889    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 26 15:19:37.777454 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.777025    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 26 15:19:37.777454 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.777145    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 26 15:19:37.777454 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.777247    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 26 15:19:37.777454 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.777429    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-c9nlk\""
Jul 26 15:19:37.777969 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.777598    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 26 15:19:37.783635 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.783594    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 26 15:19:37.825895 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.825863    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:37.876884 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.876848    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.877031 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.876903    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.877031 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.876939    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/113bfc00-208b-4a4f-9f4f-4775b2435f10-tls-assets\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.877031 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.876974    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/113bfc00-208b-4a4f-9f4f-4775b2435f10-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.877031 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.877002    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/113bfc00-208b-4a4f-9f4f-4775b2435f10-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.877287 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.877033    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-config-volume\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.877287 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.877061    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dj46\" (UniqueName: \"kubernetes.io/projected/113bfc00-208b-4a4f-9f4f-4775b2435f10-kube-api-access-2dj46\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.877287 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.877095    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.877287 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.877123    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.877287 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.877185    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.877287 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.877233    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/113bfc00-208b-4a4f-9f4f-4775b2435f10-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.877287 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.877268    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-web-config\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.877687 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.877314    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/113bfc00-208b-4a4f-9f4f-4775b2435f10-config-out\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.978152 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.978115    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.978345 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.978156    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.978345 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.978294    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.978345 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.978325    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/113bfc00-208b-4a4f-9f4f-4775b2435f10-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.978484 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.978363    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-web-config\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.978484 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.978409    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/113bfc00-208b-4a4f-9f4f-4775b2435f10-config-out\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.978484 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.978466    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.978630 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.978491    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.978630 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.978521    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/113bfc00-208b-4a4f-9f4f-4775b2435f10-tls-assets\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.978630 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.978550    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/113bfc00-208b-4a4f-9f4f-4775b2435f10-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.978630 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.978570    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/113bfc00-208b-4a4f-9f4f-4775b2435f10-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.978630 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.978597    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-config-volume\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.978630 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.978622    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dj46\" (UniqueName: \"kubernetes.io/projected/113bfc00-208b-4a4f-9f4f-4775b2435f10-kube-api-access-2dj46\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.980176 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:19:37.979459    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/113bfc00-208b-4a4f-9f4f-4775b2435f10-alertmanager-trusted-ca-bundle podName:113bfc00-208b-4a4f-9f4f-4775b2435f10 nodeName:}" failed. No retries permitted until 2026-07-26 15:19:38.479436024 +0000 UTC m=+62.769268134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/113bfc00-208b-4a4f-9f4f-4775b2435f10-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "113bfc00-208b-4a4f-9f4f-4775b2435f10") : configmap references non-existent config key: ca-bundle.crt
Jul 26 15:19:37.980408 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.980248    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/113bfc00-208b-4a4f-9f4f-4775b2435f10-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.981869 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.981840    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/113bfc00-208b-4a4f-9f4f-4775b2435f10-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.983291 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.983246    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/113bfc00-208b-4a4f-9f4f-4775b2435f10-tls-assets\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.985823 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.985781    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-config-volume\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.985925 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.985860    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.985989 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.985932    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.986046 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.986025    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.986224 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.986184    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.986528 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.986509    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-web-config\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.986599 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.986532    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/113bfc00-208b-4a4f-9f4f-4775b2435f10-config-out\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:37.988596 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:37.988549    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:38.000087 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:38.000062    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dj46\" (UniqueName: \"kubernetes.io/projected/113bfc00-208b-4a4f-9f4f-4775b2435f10-kube-api-access-2dj46\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:38.483094 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:38.483057    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/113bfc00-208b-4a4f-9f4f-4775b2435f10-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:38.484161 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:38.484128    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/113bfc00-208b-4a4f-9f4f-4775b2435f10-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:38.611286 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:38.611166    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rlzvj" event={"ID":"cab5fcb1-28bd-4602-937e-34b17c35871b","Type":"ContainerStarted","Data":"2b950fb9b0685e26256e160d607cd71ef5b1d8020d880b54e21a0f4ecf241039"}
Jul 26 15:19:38.613055 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:38.613019    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp" event={"ID":"c4a05eea-40b1-4d16-b333-72d815bc2b6b","Type":"ContainerStarted","Data":"604c13bd7704c9d7e2a2a8bb8df8a051bc7f9d316b41a3259e783c15525500a9"}
Jul 26 15:19:38.613195 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:38.613080    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp" event={"ID":"c4a05eea-40b1-4d16-b333-72d815bc2b6b","Type":"ContainerStarted","Data":"c70b14f65d5ad598a390e1ddd6fde976ed1ab22b753f180c214c94447e3c1dc1"}
Jul 26 15:19:38.613195 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:38.613100    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp" event={"ID":"c4a05eea-40b1-4d16-b333-72d815bc2b6b","Type":"ContainerStarted","Data":"eb72225017adfd551edf1b35851325a2c3b3dfa2ff407b4657552d79f9db2006"}
Jul 26 15:19:38.707676 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:38.707635    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:19:39.368892 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.368852    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:39.470421 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.470384    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:39.596824 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.596791    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:39.666036 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.665949    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-659c57d745-72nj8"]
Jul 26 15:19:39.672266 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.671986    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:39.678887 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.678393    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.680143 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.680115    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-659c57d745-72nj8"]
Jul 26 15:19:39.680632 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.680596    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-6g8hs\""
Jul 26 15:19:39.681121 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.680953    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 26 15:19:39.681121 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.680989    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 26 15:19:39.681121 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.681063    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 26 15:19:39.681316 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.681063    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1jgf4skqv5rk7\""
Jul 26 15:19:39.681316 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.681214    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 26 15:19:39.681476 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.681340    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 26 15:19:39.725909 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.725213    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:39.794974 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.794924    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e738e949-e8ce-44a5-920a-00aacdb86c26-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.794974 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.794973    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e738e949-e8ce-44a5-920a-00aacdb86c26-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.795244 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.795008    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e738e949-e8ce-44a5-920a-00aacdb86c26-secret-thanos-querier-tls\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.795244 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.795056    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4t84\" (UniqueName: \"kubernetes.io/projected/e738e949-e8ce-44a5-920a-00aacdb86c26-kube-api-access-g4t84\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.795244 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.795093    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e738e949-e8ce-44a5-920a-00aacdb86c26-secret-grpc-tls\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.795244 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.795119    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e738e949-e8ce-44a5-920a-00aacdb86c26-metrics-client-ca\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.795244 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.795158    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e738e949-e8ce-44a5-920a-00aacdb86c26-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.795244 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.795218    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e738e949-e8ce-44a5-920a-00aacdb86c26-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.896166 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.896123    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e738e949-e8ce-44a5-920a-00aacdb86c26-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.896360 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.896185    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e738e949-e8ce-44a5-920a-00aacdb86c26-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.896360 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.896266    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e738e949-e8ce-44a5-920a-00aacdb86c26-secret-thanos-querier-tls\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.896360 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.896310    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4t84\" (UniqueName: \"kubernetes.io/projected/e738e949-e8ce-44a5-920a-00aacdb86c26-kube-api-access-g4t84\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.896360 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.896339    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e738e949-e8ce-44a5-920a-00aacdb86c26-secret-grpc-tls\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.896568 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.896370    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e738e949-e8ce-44a5-920a-00aacdb86c26-metrics-client-ca\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.896568 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.896416    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e738e949-e8ce-44a5-920a-00aacdb86c26-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.896568 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.896464    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e738e949-e8ce-44a5-920a-00aacdb86c26-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.897252 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.897180    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e738e949-e8ce-44a5-920a-00aacdb86c26-metrics-client-ca\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.899326 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.899299    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e738e949-e8ce-44a5-920a-00aacdb86c26-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.899455 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.899384    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e738e949-e8ce-44a5-920a-00aacdb86c26-secret-grpc-tls\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.899455 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.899385    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e738e949-e8ce-44a5-920a-00aacdb86c26-secret-thanos-querier-tls\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.899541 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.899483    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e738e949-e8ce-44a5-920a-00aacdb86c26-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.899541 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.899482    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e738e949-e8ce-44a5-920a-00aacdb86c26-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.899771 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.899750    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e738e949-e8ce-44a5-920a-00aacdb86c26-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.904529 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.904508    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4t84\" (UniqueName: \"kubernetes.io/projected/e738e949-e8ce-44a5-920a-00aacdb86c26-kube-api-access-g4t84\") pod \"thanos-querier-659c57d745-72nj8\" (UID: \"e738e949-e8ce-44a5-920a-00aacdb86c26\") " pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:39.990561 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:39.990474    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:40.172609 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.172575    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:40.206773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.206743    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-5r4xv"]
Jul 26 15:19:40.213185 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.213152    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-5r4xv"
Jul 26 15:19:40.215108 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.215080    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 26 15:19:40.215108 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.215099    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 26 15:19:40.215319 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.215266    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-5zw9v\""
Jul 26 15:19:40.220352 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.220329    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-5r4xv"]
Jul 26 15:19:40.240553 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.240511    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:40.300148 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.300060    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45vtl\" (UniqueName: \"kubernetes.io/projected/43e1c401-0b5d-415c-ab0b-f91b4303dcf9-kube-api-access-45vtl\") pod \"downloads-7c97dc4555-5r4xv\" (UID: \"43e1c401-0b5d-415c-ab0b-f91b4303dcf9\") " pod="openshift-console/downloads-7c97dc4555-5r4xv"
Jul 26 15:19:40.315405 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.315364    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:40.401449 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.401395    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45vtl\" (UniqueName: \"kubernetes.io/projected/43e1c401-0b5d-415c-ab0b-f91b4303dcf9-kube-api-access-45vtl\") pod \"downloads-7c97dc4555-5r4xv\" (UID: \"43e1c401-0b5d-415c-ab0b-f91b4303dcf9\") " pod="openshift-console/downloads-7c97dc4555-5r4xv"
Jul 26 15:19:40.412231 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.412185    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45vtl\" (UniqueName: \"kubernetes.io/projected/43e1c401-0b5d-415c-ab0b-f91b4303dcf9-kube-api-access-45vtl\") pod \"downloads-7c97dc4555-5r4xv\" (UID: \"43e1c401-0b5d-415c-ab0b-f91b4303dcf9\") " pod="openshift-console/downloads-7c97dc4555-5r4xv"
Jul 26 15:19:40.524797 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.524768    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-5r4xv"
Jul 26 15:19:40.526746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.526718    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:40.894780 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.894745    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 26 15:19:40.902139 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:19:40.901738    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod113bfc00_208b_4a4f_9f4f_4775b2435f10.slice/crio-18067642bdb9413ec84cd90bfccb7c4b37af269f2d36b5ee383cf2a62cbafdf0 WatchSource:0}: Error finding container 18067642bdb9413ec84cd90bfccb7c4b37af269f2d36b5ee383cf2a62cbafdf0: Status 404 returned error can't find the container with id 18067642bdb9413ec84cd90bfccb7c4b37af269f2d36b5ee383cf2a62cbafdf0
Jul 26 15:19:40.903174 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.903142    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:40.906638 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.906611    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs\") pod \"network-metrics-daemon-p8dd2\" (UID: \"63512a39-e727-488a-96ea-e93a866aad7d\") " pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:40.908411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.908387    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 26 15:19:40.908514 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.908476    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-5r4xv"]
Jul 26 15:19:40.912004 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:19:40.911974    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43e1c401_0b5d_415c_ab0b_f91b4303dcf9.slice/crio-7ef8c34a769e5c38a662d8ae067fb4ebd6a4a81f93efb6360bd449db97681f9e WatchSource:0}: Error finding container 7ef8c34a769e5c38a662d8ae067fb4ebd6a4a81f93efb6360bd449db97681f9e: Status 404 returned error can't find the container with id 7ef8c34a769e5c38a662d8ae067fb4ebd6a4a81f93efb6360bd449db97681f9e
Jul 26 15:19:40.922779 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.922125    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63512a39-e727-488a-96ea-e93a866aad7d-metrics-certs\") pod \"network-metrics-daemon-p8dd2\" (UID: \"63512a39-e727-488a-96ea-e93a866aad7d\") " pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:40.927179 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.927143    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-659c57d745-72nj8"]
Jul 26 15:19:40.965770 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.965710    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:40.996464 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.996430    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-775b87ff9b-hzhj9"]
Jul 26 15:19:40.998581 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:40.998558    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.001126 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.001098    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 26 15:19:41.001800 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.001436    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 26 15:19:41.001800 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.001140    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-27wvv\""
Jul 26 15:19:41.001800 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.001162    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 26 15:19:41.001800 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.001253    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-81s111m6gffnd\""
Jul 26 15:19:41.001800 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.001165    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 26 15:19:41.007920 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.007892    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret\") pod \"global-pull-secret-syncer-6g6lj\" (UID: \"5645fd38-99a5-4fa7-bf50-bf044c0444b6\") " pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:41.010288 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.010264    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 26 15:19:41.010926 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.010905    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-775b87ff9b-hzhj9"]
Jul 26 15:19:41.021344 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.021310    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5645fd38-99a5-4fa7-bf50-bf044c0444b6-original-pull-secret\") pod \"global-pull-secret-syncer-6g6lj\" (UID: \"5645fd38-99a5-4fa7-bf50-bf044c0444b6\") " pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:41.109335 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.109299    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q8vl\" (UniqueName: \"kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl\") pod \"network-check-target-gztc2\" (UID: \"b003fa80-7770-4f06-9917-926f86feebf4\") " pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:41.109506 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.109353    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e2e42628-3aa6-421f-b3c5-06ed360daa81-secret-metrics-server-client-certs\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.109506 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.109371    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2e42628-3aa6-421f-b3c5-06ed360daa81-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.109506 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.109399    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e2e42628-3aa6-421f-b3c5-06ed360daa81-audit-log\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.109506 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.109467    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e2e42628-3aa6-421f-b3c5-06ed360daa81-secret-metrics-server-tls\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.109668 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.109512    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e2e42628-3aa6-421f-b3c5-06ed360daa81-metrics-server-audit-profiles\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.109668 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.109556    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf757\" (UniqueName: \"kubernetes.io/projected/e2e42628-3aa6-421f-b3c5-06ed360daa81-kube-api-access-xf757\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.109668 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.109585    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2e42628-3aa6-421f-b3c5-06ed360daa81-client-ca-bundle\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.111710 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.111688    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 26 15:19:41.121947 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.121914    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 26 15:19:41.132782 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.132752    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q8vl\" (UniqueName: \"kubernetes.io/projected/b003fa80-7770-4f06-9917-926f86feebf4-kube-api-access-5q8vl\") pod \"network-check-target-gztc2\" (UID: \"b003fa80-7770-4f06-9917-926f86feebf4\") " pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:41.202010 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.201976    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6xwws\""
Jul 26 15:19:41.205545 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.205528    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g6lj"
Jul 26 15:19:41.210361 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.210328    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8dd2"
Jul 26 15:19:41.210662 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.210632    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e2e42628-3aa6-421f-b3c5-06ed360daa81-secret-metrics-server-tls\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.210767 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.210680    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e2e42628-3aa6-421f-b3c5-06ed360daa81-metrics-server-audit-profiles\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.210767 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.210715    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf757\" (UniqueName: \"kubernetes.io/projected/e2e42628-3aa6-421f-b3c5-06ed360daa81-kube-api-access-xf757\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.210767 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.210745    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2e42628-3aa6-421f-b3c5-06ed360daa81-client-ca-bundle\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.210916 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.210829    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e2e42628-3aa6-421f-b3c5-06ed360daa81-secret-metrics-server-client-certs\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.210916 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.210857    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2e42628-3aa6-421f-b3c5-06ed360daa81-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.210916 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.210902    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e2e42628-3aa6-421f-b3c5-06ed360daa81-audit-log\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.211714 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.211360    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e2e42628-3aa6-421f-b3c5-06ed360daa81-audit-log\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.211994 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.211968    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e2e42628-3aa6-421f-b3c5-06ed360daa81-metrics-server-audit-profiles\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.211994 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.211983    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2e42628-3aa6-421f-b3c5-06ed360daa81-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.213656 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.213451    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bcjwm\""
Jul 26 15:19:41.213656 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.213554    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e2e42628-3aa6-421f-b3c5-06ed360daa81-secret-metrics-server-tls\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.213844 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.213802    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e2e42628-3aa6-421f-b3c5-06ed360daa81-secret-metrics-server-client-certs\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.214133 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.214109    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2e42628-3aa6-421f-b3c5-06ed360daa81-client-ca-bundle\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.218459 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.218436    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf757\" (UniqueName: \"kubernetes.io/projected/e2e42628-3aa6-421f-b3c5-06ed360daa81-kube-api-access-xf757\") pod \"metrics-server-775b87ff9b-hzhj9\" (UID: \"e2e42628-3aa6-421f-b3c5-06ed360daa81\") " pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.222161 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.222138    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:41.313182 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.313102    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:41.334156 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.334128    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:19:41.357449 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.357285    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6g6lj"]
Jul 26 15:19:41.363545 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:19:41.363517    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5645fd38_99a5_4fa7_bf50_bf044c0444b6.slice/crio-7ebfd0fc8f753c17a5482b5351640019885ec828f415063a1bf7c2dae4c23ef9 WatchSource:0}: Error finding container 7ebfd0fc8f753c17a5482b5351640019885ec828f415063a1bf7c2dae4c23ef9: Status 404 returned error can't find the container with id 7ebfd0fc8f753c17a5482b5351640019885ec828f415063a1bf7c2dae4c23ef9
Jul 26 15:19:41.384326 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.384063    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p8dd2"]
Jul 26 15:19:41.391417 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:19:41.391375    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63512a39_e727_488a_96ea_e93a866aad7d.slice/crio-dfec8d1905a376df8c2b31bf5347585977c0435de7e963d88df969df478ba6db WatchSource:0}: Error finding container dfec8d1905a376df8c2b31bf5347585977c0435de7e963d88df969df478ba6db: Status 404 returned error can't find the container with id dfec8d1905a376df8c2b31bf5347585977c0435de7e963d88df969df478ba6db
Jul 26 15:19:41.414029 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.413997    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gztc2"]
Jul 26 15:19:41.420443 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:19:41.419669    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb003fa80_7770_4f06_9917_926f86feebf4.slice/crio-c28373a273bdbd0b3f475d5dd01dcc91eb14d70186ed66f08186b1c8373ca13a WatchSource:0}: Error finding container c28373a273bdbd0b3f475d5dd01dcc91eb14d70186ed66f08186b1c8373ca13a: Status 404 returned error can't find the container with id c28373a273bdbd0b3f475d5dd01dcc91eb14d70186ed66f08186b1c8373ca13a
Jul 26 15:19:41.454656 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.454566    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:41.469381 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.468003    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-vtjpr"]
Jul 26 15:19:41.471225 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.471122    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vtjpr"
Jul 26 15:19:41.477597 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.475163    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-8ll5t\""
Jul 26 15:19:41.477597 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.476324    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 26 15:19:41.483483 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.482750    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-vtjpr"]
Jul 26 15:19:41.516073 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.515893    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtpxh\" (UniqueName: \"kubernetes.io/projected/57e0df8f-7130-481e-998b-89f6e93e8b01-kube-api-access-jtpxh\") pod \"monitoring-plugin-68b7fc9574-vtjpr\" (UID: \"57e0df8f-7130-481e-998b-89f6e93e8b01\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vtjpr"
Jul 26 15:19:41.516073 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.515954    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/57e0df8f-7130-481e-998b-89f6e93e8b01-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-vtjpr\" (UID: \"57e0df8f-7130-481e-998b-89f6e93e8b01\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vtjpr"
Jul 26 15:19:41.518051 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.517957    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:41.568605 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.568564    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-775b87ff9b-hzhj9"]
Jul 26 15:19:41.573101 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:19:41.573066    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2e42628_3aa6_421f_b3c5_06ed360daa81.slice/crio-e3cde9ec50fbf3aad3b282febb76984e76ffd42d36aadfe533db55aeca6ef0bc WatchSource:0}: Error finding container e3cde9ec50fbf3aad3b282febb76984e76ffd42d36aadfe533db55aeca6ef0bc: Status 404 returned error can't find the container with id e3cde9ec50fbf3aad3b282febb76984e76ffd42d36aadfe533db55aeca6ef0bc
Jul 26 15:19:41.596398 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.596364    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:41.616558 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.616523    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtpxh\" (UniqueName: \"kubernetes.io/projected/57e0df8f-7130-481e-998b-89f6e93e8b01-kube-api-access-jtpxh\") pod \"monitoring-plugin-68b7fc9574-vtjpr\" (UID: \"57e0df8f-7130-481e-998b-89f6e93e8b01\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vtjpr"
Jul 26 15:19:41.616744 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.616584    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/57e0df8f-7130-481e-998b-89f6e93e8b01-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-vtjpr\" (UID: \"57e0df8f-7130-481e-998b-89f6e93e8b01\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vtjpr"
Jul 26 15:19:41.619784 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.619728    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/57e0df8f-7130-481e-998b-89f6e93e8b01-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-vtjpr\" (UID: \"57e0df8f-7130-481e-998b-89f6e93e8b01\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vtjpr"
Jul 26 15:19:41.625069 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.625018    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtpxh\" (UniqueName: \"kubernetes.io/projected/57e0df8f-7130-481e-998b-89f6e93e8b01-kube-api-access-jtpxh\") pod \"monitoring-plugin-68b7fc9574-vtjpr\" (UID: \"57e0df8f-7130-481e-998b-89f6e93e8b01\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vtjpr"
Jul 26 15:19:41.628678 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.628614    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-5r4xv" event={"ID":"43e1c401-0b5d-415c-ab0b-f91b4303dcf9","Type":"ContainerStarted","Data":"7ef8c34a769e5c38a662d8ae067fb4ebd6a4a81f93efb6360bd449db97681f9e"}
Jul 26 15:19:41.631568 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.631539    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"113bfc00-208b-4a4f-9f4f-4775b2435f10","Type":"ContainerStarted","Data":"18067642bdb9413ec84cd90bfccb7c4b37af269f2d36b5ee383cf2a62cbafdf0"}
Jul 26 15:19:41.636536 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.636506    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h" event={"ID":"70542744-fd1a-4471-b33d-80747ea33774","Type":"ContainerStarted","Data":"26eac0c2cf141820fc8ea280f46fa7613610bb1d94b318780464da541a770d6c"}
Jul 26 15:19:41.636632 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.636569    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h" event={"ID":"70542744-fd1a-4471-b33d-80747ea33774","Type":"ContainerStarted","Data":"5ddb54d7faf0987db127158c653b86a1e6146b7e257b0fc2f4e40390329a7a59"}
Jul 26 15:19:41.636632 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.636598    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h" event={"ID":"70542744-fd1a-4471-b33d-80747ea33774","Type":"ContainerStarted","Data":"f96eeabf5660b4c4aa2d0e72668d40c0823018724908c205c6c748d165d705ed"}
Jul 26 15:19:41.640557 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.640483    2559 generic.go:356] "Generic (PLEG): container finished" podID="cab5fcb1-28bd-4602-937e-34b17c35871b" containerID="9fd97d934e053429ddfe2465a66f4c05a8d7de6d2cff827111402a5d914993f8" exitCode=0
Jul 26 15:19:41.640680 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.640565    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rlzvj" event={"ID":"cab5fcb1-28bd-4602-937e-34b17c35871b","Type":"ContainerDied","Data":"9fd97d934e053429ddfe2465a66f4c05a8d7de6d2cff827111402a5d914993f8"}
Jul 26 15:19:41.642979 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.642834    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p8dd2" event={"ID":"63512a39-e727-488a-96ea-e93a866aad7d","Type":"ContainerStarted","Data":"dfec8d1905a376df8c2b31bf5347585977c0435de7e963d88df969df478ba6db"}
Jul 26 15:19:41.645246 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.645093    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-659c57d745-72nj8" event={"ID":"e738e949-e8ce-44a5-920a-00aacdb86c26","Type":"ContainerStarted","Data":"5734a08c69143e971c5dcf93d9737344b03f306d94e09ec0e482c81cd251720a"}
Jul 26 15:19:41.648507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.648471    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s7hbj" event={"ID":"7d91f8c8-c702-4f74-a29e-af54043a79d5","Type":"ContainerStarted","Data":"82f2e2bedfb69dbeebf30704cfb87c1a99ded73be30d0292d5e517f3d0e2706c"}
Jul 26 15:19:41.652755 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.652726    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp" event={"ID":"c4a05eea-40b1-4d16-b333-72d815bc2b6b","Type":"ContainerStarted","Data":"b378929435883cb9cfa277e9afa31131befe3372ba3e3154a07ce41c04a893ee"}
Jul 26 15:19:41.659563 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.659096    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7h29h" podStartSLOduration=2.120124229 podStartE2EDuration="5.659075338s" totalImagesPullingTime="3.538951109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:19:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:19:41.654559336 +0000 UTC m=+65.944391455" watchObservedRunningTime="2026-07-26 15:19:41.659075338 +0000 UTC m=+65.948907458"
Jul 26 15:19:41.660246 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.660175    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6g6lj" event={"ID":"5645fd38-99a5-4fa7-bf50-bf044c0444b6","Type":"ContainerStarted","Data":"7ebfd0fc8f753c17a5482b5351640019885ec828f415063a1bf7c2dae4c23ef9"}
Jul 26 15:19:41.661726 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.661668    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9" event={"ID":"e2e42628-3aa6-421f-b3c5-06ed360daa81","Type":"ContainerStarted","Data":"e3cde9ec50fbf3aad3b282febb76984e76ffd42d36aadfe533db55aeca6ef0bc"}
Jul 26 15:19:41.663403 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.663338    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gztc2" event={"ID":"b003fa80-7770-4f06-9917-926f86feebf4","Type":"ContainerStarted","Data":"c28373a273bdbd0b3f475d5dd01dcc91eb14d70186ed66f08186b1c8373ca13a"}
Jul 26 15:19:41.695956 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.695582    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-s7hbj" podStartSLOduration=2.300297413 podStartE2EDuration="16.695558334s" totalImagesPullingTime="14.395260921s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:19:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:19:41.693304937 +0000 UTC m=+65.983137058" watchObservedRunningTime="2026-07-26 15:19:41.695558334 +0000 UTC m=+65.985390631"
Jul 26 15:19:41.725474 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.723425    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:41.727597 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.726978    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cjjgp" podStartSLOduration=2.925793387 podStartE2EDuration="5.726956159s" totalImagesPullingTime="2.801162772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:19:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:19:41.714616857 +0000 UTC m=+66.004448988" watchObservedRunningTime="2026-07-26 15:19:41.726956159 +0000 UTC m=+66.016788280"
Jul 26 15:19:41.777943 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.777878    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:41.791598 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.791327    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vtjpr"
Jul 26 15:19:41.926739 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.926673    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:41.989894 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.989813    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-vtjpr"]
Jul 26 15:19:41.994644 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:41.994335    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:41.999568 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:19:41.999536    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57e0df8f_7130_481e_998b_89f6e93e8b01.slice/crio-8e2df76a5cb3d1ace34237565c85ee525a3e4ef767065915986a0b062da1a9db WatchSource:0}: Error finding container 8e2df76a5cb3d1ace34237565c85ee525a3e4ef767065915986a0b062da1a9db: Status 404 returned error can't find the container with id 8e2df76a5cb3d1ace34237565c85ee525a3e4ef767065915986a0b062da1a9db
Jul 26 15:19:42.107710 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.107584    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:42.374188 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.374105    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:42.522490 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.522347    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:42.576967 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.576855    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:42.680414 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.680317    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vtjpr" event={"ID":"57e0df8f-7130-481e-998b-89f6e93e8b01","Type":"ContainerStarted","Data":"8e2df76a5cb3d1ace34237565c85ee525a3e4ef767065915986a0b062da1a9db"}
Jul 26 15:19:42.687072 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.685864    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rlzvj" event={"ID":"cab5fcb1-28bd-4602-937e-34b17c35871b","Type":"ContainerStarted","Data":"037092e0a092180485cb164ed8e03f726392ce350071fed828a6dac76c1162ab"}
Jul 26 15:19:42.687072 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.685926    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rlzvj" event={"ID":"cab5fcb1-28bd-4602-937e-34b17c35871b","Type":"ContainerStarted","Data":"a1891d3b0850e9538da4f7dcbad9258d69c429d9474531a50d13195993aa6596"}
Jul 26 15:19:42.704113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.703579    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rlzvj" podStartSLOduration=3.526276257 podStartE2EDuration="6.703555683s" totalImagesPullingTime="3.06130603s" totalInitContainerRuntime="115.973396ms" isStatefulPod=true podCreationTimestamp="2026-07-26 15:19:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:19:42.702475015 +0000 UTC m=+66.992307136" watchObservedRunningTime="2026-07-26 15:19:42.703555683 +0000 UTC m=+66.993387805"
Jul 26 15:19:42.957884 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.957792    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 26 15:19:42.958332 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.957790    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:42.964339 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.962882    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:42.966297 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.965625    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 26 15:19:42.966297 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.965649    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 26 15:19:42.966297 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.965882    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 26 15:19:42.970274 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.967164    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 26 15:19:42.970274 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.967430    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 26 15:19:42.970274 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.967704    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 26 15:19:42.970274 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.967881    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4fr2e752lc0r4\""
Jul 26 15:19:42.970274 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.968144    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 26 15:19:42.970274 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.968376    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pv29c\""
Jul 26 15:19:42.970274 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.968558    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 26 15:19:42.970274 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.968736    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 26 15:19:42.970274 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.969007    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 26 15:19:42.970274 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.969194    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 26 15:19:42.970274 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.969390    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 26 15:19:42.988776 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.975984    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 26 15:19:42.988776 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.985591    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 26 15:19:42.996008 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:42.990999    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 26 15:19:43.036546 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.036508    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc5bea94-f396-4d67-ae50-b070a870a64e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.036734 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.036559    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.036734 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.036594    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-config\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.036734 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.036618    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.036734 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.036639    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.036734 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.036708    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc5bea94-f396-4d67-ae50-b070a870a64e-config-out\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.036734 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.036732    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.037021 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.036770    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.037021 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.036795    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.037021 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.036826    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.037576 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.037152    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.037576 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.037287    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.037576 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.037354    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.037576 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.037437    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.037576 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.037463    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.037576 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.037506    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.037576 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.037540    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.037576 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.037570    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.038022 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.037600    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-web-config\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.038022 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.037622    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnc9q\" (UniqueName: \"kubernetes.io/projected/bc5bea94-f396-4d67-ae50-b070a870a64e-kube-api-access-wnc9q\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.140136 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.140036    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.140136 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.140083    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.148468 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.142253    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.148468 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.142253    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.148712 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.148597    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc5bea94-f396-4d67-ae50-b070a870a64e-config-out\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.148712 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.148656    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.148712 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.148695    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.148852 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.148727    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.148852 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.148761    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.148852 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.148796    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.148852 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.148836    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.149035 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.148870    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.149035 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.148946    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.150273 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.149393    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.150273 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.149470    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.150273 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.149525    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.150273 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.149571    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.150273 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.149601    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-web-config\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.150273 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.149638    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnc9q\" (UniqueName: \"kubernetes.io/projected/bc5bea94-f396-4d67-ae50-b070a870a64e-kube-api-access-wnc9q\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.150273 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.149675    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc5bea94-f396-4d67-ae50-b070a870a64e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.150273 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.149719    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.150273 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.149762    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-config\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.150767 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.150545    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.151691 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.151101    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.152645 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.152619    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.155290 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.155263    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.167601 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.165070    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.168307 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.167806    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.168307 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.168090    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.169264 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.168605    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.169264 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.168670    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.169264 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.169118    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.169439 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.169342    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-web-config\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.169744 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.169564    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.169744 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.169693    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.173040 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.173012    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc5bea94-f396-4d67-ae50-b070a870a64e-config-out\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.173281 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.173243    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc5bea94-f396-4d67-ae50-b070a870a64e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.173523 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.173504    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-config\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.175917 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.175138    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnc9q\" (UniqueName: \"kubernetes.io/projected/bc5bea94-f396-4d67-ae50-b070a870a64e-kube-api-access-wnc9q\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.179262 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.179194    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.300967 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.300591    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:19:43.596424 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.595477    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-drfng"
Jul 26 15:19:43.701531 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:43.701362    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:44.318063 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:44.317772    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:44.418146 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:44.418064    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:44.525183 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:44.525055    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:44.625612 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:44.625517    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:47.133570 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:47.133517    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:47.236514 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:47.236475    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:47.258567 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:47.258430    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 26 15:19:48.786359 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:19:48.786321    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc5bea94_f396_4d67_ae50_b070a870a64e.slice/crio-cfd9d6fb61850f58fc4a961331d450d28a0cce110bbda60de427d4fc7e4f00e6 WatchSource:0}: Error finding container cfd9d6fb61850f58fc4a961331d450d28a0cce110bbda60de427d4fc7e4f00e6: Status 404 returned error can't find the container with id cfd9d6fb61850f58fc4a961331d450d28a0cce110bbda60de427d4fc7e4f00e6
Jul 26 15:19:49.121177 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.121147    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:49.220646 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.220612    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:49.727140 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.727097    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6g6lj" event={"ID":"5645fd38-99a5-4fa7-bf50-bf044c0444b6","Type":"ContainerStarted","Data":"d8249951d04ba756a1b4c10a5677bbf266de5ca3316c155379f4f9c9c0832b2e"}
Jul 26 15:19:49.729129 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.729096    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9" event={"ID":"e2e42628-3aa6-421f-b3c5-06ed360daa81","Type":"ContainerStarted","Data":"10b42d68ecd9202b09573cecf069828f05e216a416092822067eb4bfb964a782"}
Jul 26 15:19:49.731897 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.731849    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gztc2" event={"ID":"b003fa80-7770-4f06-9917-926f86feebf4","Type":"ContainerStarted","Data":"fc4a3363430bb1d3a6253dcb1053a2b645cbeb2bea7fafcf666d82cc0e1d2b39"}
Jul 26 15:19:49.732004 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.731957    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:19:49.736233 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.736193    2559 generic.go:356] "Generic (PLEG): container finished" podID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerID="31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c" exitCode=0
Jul 26 15:19:49.736342 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.736296    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc5bea94-f396-4d67-ae50-b070a870a64e","Type":"ContainerDied","Data":"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c"}
Jul 26 15:19:49.736407 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.736355    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc5bea94-f396-4d67-ae50-b070a870a64e","Type":"ContainerStarted","Data":"cfd9d6fb61850f58fc4a961331d450d28a0cce110bbda60de427d4fc7e4f00e6"}
Jul 26 15:19:49.739046 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.739025    2559 generic.go:356] "Generic (PLEG): container finished" podID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerID="a0419517f74b9ff726e51e781bfa65a31518ad5cbba77658ffe50c77c4647273" exitCode=0
Jul 26 15:19:49.739129 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.739095    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"113bfc00-208b-4a4f-9f4f-4775b2435f10","Type":"ContainerDied","Data":"a0419517f74b9ff726e51e781bfa65a31518ad5cbba77658ffe50c77c4647273"}
Jul 26 15:19:49.742937 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.742901    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vtjpr" event={"ID":"57e0df8f-7130-481e-998b-89f6e93e8b01","Type":"ContainerStarted","Data":"60fbdcf02b33fe8d02f43af79e843be6d9ad010499467e38f6e9fffece842559"}
Jul 26 15:19:49.743929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.743224    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6g6lj" podStartSLOduration=65.251615766 podStartE2EDuration="1m12.743191275s" totalImagesPullingTime="7.491575509s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:18:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:19:49.742060176 +0000 UTC m=+74.031892295" watchObservedRunningTime="2026-07-26 15:19:49.743191275 +0000 UTC m=+74.033023395"
Jul 26 15:19:49.743929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.743675    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vtjpr"
Jul 26 15:19:49.747010 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.746988    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p8dd2" event={"ID":"63512a39-e727-488a-96ea-e93a866aad7d","Type":"ContainerStarted","Data":"79b3e20fd2c3ea44024854e8e823500ce9b6a5bff39e9d8229822fe1e7568881"}
Jul 26 15:19:49.747281 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.747193    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p8dd2" event={"ID":"63512a39-e727-488a-96ea-e93a866aad7d","Type":"ContainerStarted","Data":"456834bbe6a7fb191568a0ed1c5efbe51a7aa71ba1f2aa9e13ea2571d50a0831"}
Jul 26 15:19:49.751181 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.750913    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-659c57d745-72nj8" event={"ID":"e738e949-e8ce-44a5-920a-00aacdb86c26","Type":"ContainerStarted","Data":"31893488f422aaa911936ec4acb2b1a30e7374db389a362acabf6a46bdedbf5b"}
Jul 26 15:19:49.751181 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.750980    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-659c57d745-72nj8" event={"ID":"e738e949-e8ce-44a5-920a-00aacdb86c26","Type":"ContainerStarted","Data":"c0b1e387174134767ed097f73d53b7502919b8d6269a52007285ec8a1161835b"}
Jul 26 15:19:49.751181 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.751020    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-659c57d745-72nj8" event={"ID":"e738e949-e8ce-44a5-920a-00aacdb86c26","Type":"ContainerStarted","Data":"eb286ad2cf91eade8162dbccedae9fac735a82d25b96382c20d6c3a0aee06db9"}
Jul 26 15:19:49.752079 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.751849    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vtjpr"
Jul 26 15:19:49.756850 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.756808    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gztc2" podStartSLOduration=65.339584519 podStartE2EDuration="1m12.756790826s" totalImagesPullingTime="7.417206307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:18:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:19:49.754867179 +0000 UTC m=+74.044699295" watchObservedRunningTime="2026-07-26 15:19:49.756790826 +0000 UTC m=+74.046622946"
Jul 26 15:19:49.847147 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.846640    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9" podStartSLOduration=2.623627683 podStartE2EDuration="9.846619022s" totalImagesPullingTime="7.222991339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:19:49.845607254 +0000 UTC m=+74.135439386" watchObservedRunningTime="2026-07-26 15:19:49.846619022 +0000 UTC m=+74.136451142"
Jul 26 15:19:49.864805 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.864749    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-vtjpr" podStartSLOduration=2.0717485350000002 podStartE2EDuration="8.864726995s" totalImagesPullingTime="6.79297846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:19:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:19:49.863719432 +0000 UTC m=+74.153551552" watchObservedRunningTime="2026-07-26 15:19:49.864726995 +0000 UTC m=+74.154559116"
Jul 26 15:19:49.878609 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.878555    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-p8dd2" podStartSLOduration=65.481259582 podStartE2EDuration="1m12.878534243s" totalImagesPullingTime="7.397274661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:18:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:19:49.878465056 +0000 UTC m=+74.168297176" watchObservedRunningTime="2026-07-26 15:19:49.878534243 +0000 UTC m=+74.168366365"
Jul 26 15:19:49.965303 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:49.965274    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:50.066870 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:50.066197    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:50.761742 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:50.761705    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-659c57d745-72nj8" event={"ID":"e738e949-e8ce-44a5-920a-00aacdb86c26","Type":"ContainerStarted","Data":"a67923b4168f41aaaa956ab6b5b3b7a66570f4cc06fc09abc0414ddf61c98119"}
Jul 26 15:19:50.762011 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:50.761780    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-659c57d745-72nj8" event={"ID":"e738e949-e8ce-44a5-920a-00aacdb86c26","Type":"ContainerStarted","Data":"543c098e896a12e9628b88a3d4d46d129d0c81f36617ca707c5162efa6f2e2b0"}
Jul 26 15:19:50.762011 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:50.761820    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-659c57d745-72nj8" event={"ID":"e738e949-e8ce-44a5-920a-00aacdb86c26","Type":"ContainerStarted","Data":"fc7fb801afd6a5171fd19c1c46bc4bed9ba5f3d7330893461049d067b1fa22eb"}
Jul 26 15:19:50.793394 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:50.793333    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-659c57d745-72nj8" podStartSLOduration=3.150911611 podStartE2EDuration="11.793308676s" totalImagesPullingTime="8.642397065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:19:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:19:50.788781214 +0000 UTC m=+75.078613326" watchObservedRunningTime="2026-07-26 15:19:50.793308676 +0000 UTC m=+75.083140798"
Jul 26 15:19:51.775411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:51.774889    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"113bfc00-208b-4a4f-9f4f-4775b2435f10","Type":"ContainerStarted","Data":"15f04bace33bf09f83779b409a3cd51f9b6da4ab2c56c863f3eca2b1de631346"}
Jul 26 15:19:51.775411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:51.774985    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"113bfc00-208b-4a4f-9f4f-4775b2435f10","Type":"ContainerStarted","Data":"465a95a09a61b6c76da7caa3b65b0de52c7b1ae94a2b1a95f048f8f12b834147"}
Jul 26 15:19:51.775411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:51.775012    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"113bfc00-208b-4a4f-9f4f-4775b2435f10","Type":"ContainerStarted","Data":"4f78c2bb02ab9d9710d931a9434a5df1a2887c0c248d76cdd3e3e9d7bd78d5df"}
Jul 26 15:19:51.775411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:51.775036    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"113bfc00-208b-4a4f-9f4f-4775b2435f10","Type":"ContainerStarted","Data":"9e487eb5b2c258ac09a68740d962a7a053a02dd85f9a0a63b27c9ad2c388974a"}
Jul 26 15:19:51.775411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:51.775326    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:52.784302 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:52.784263    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"113bfc00-208b-4a4f-9f4f-4775b2435f10","Type":"ContainerStarted","Data":"a96260a488d22c700aeb80a2ddd7943f136c1befa94fb698da25406eb510e8c0"}
Jul 26 15:19:52.784770 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:52.784347    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"113bfc00-208b-4a4f-9f4f-4775b2435f10","Type":"ContainerStarted","Data":"c1dd555713c53fe339125f768d32809bed3d95b920dcbd1138a1f658970134c0"}
Jul 26 15:19:52.813577 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:52.812844    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=7.928773832 podStartE2EDuration="15.812819455s" totalImagesPullingTime="7.848838225s" totalInitContainerRuntime="35.207398ms" isStatefulPod=false podCreationTimestamp="2026-07-26 15:19:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:19:52.810069339 +0000 UTC m=+77.099901459" watchObservedRunningTime="2026-07-26 15:19:52.812819455 +0000 UTC m=+77.102651578"
Jul 26 15:19:53.965000 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:53.964961    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:54.064388 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:54.064352    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:56.201439 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:56.201404    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:56.254343 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:56.254305    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:57.793130 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:57.792846    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-659c57d745-72nj8"
Jul 26 15:19:57.990568 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:57.990529    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:19:58.093331 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:19:58.093299    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:00.527454 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:00.527410    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:00.578529 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:00.578501    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:01.334364 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:01.334331    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:20:01.334495 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:01.334425    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:20:01.831468 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:01.831418    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc5bea94-f396-4d67-ae50-b070a870a64e","Type":"ContainerStarted","Data":"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1"}
Jul 26 15:20:01.831915 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:01.831515    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc5bea94-f396-4d67-ae50-b070a870a64e","Type":"ContainerStarted","Data":"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d"}
Jul 26 15:20:01.831915 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:01.831560    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc5bea94-f396-4d67-ae50-b070a870a64e","Type":"ContainerStarted","Data":"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421"}
Jul 26 15:20:01.831915 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:01.831598    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc5bea94-f396-4d67-ae50-b070a870a64e","Type":"ContainerStarted","Data":"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2"}
Jul 26 15:20:01.831915 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:01.831634    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc5bea94-f396-4d67-ae50-b070a870a64e","Type":"ContainerStarted","Data":"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b"}
Jul 26 15:20:01.832932 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:01.832905    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-5r4xv" event={"ID":"43e1c401-0b5d-415c-ab0b-f91b4303dcf9","Type":"ContainerStarted","Data":"6425753f20049bea857c9235a2a2a717de9fef2dd4488a602702db3990a7ac5b"}
Jul 26 15:20:01.833136 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:01.833116    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-5r4xv"
Jul 26 15:20:01.850025 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:01.849975    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-5r4xv" podStartSLOduration=1.489223049 podStartE2EDuration="21.8499586s" totalImagesPullingTime="20.360735551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:20:01.847291117 +0000 UTC m=+86.137123237" watchObservedRunningTime="2026-07-26 15:20:01.8499586 +0000 UTC m=+86.139790773"
Jul 26 15:20:01.851823 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:01.851802    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-5r4xv"
Jul 26 15:20:02.842772 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:02.842727    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc5bea94-f396-4d67-ae50-b070a870a64e","Type":"ContainerStarted","Data":"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9"}
Jul 26 15:20:02.869829 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:02.869770    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=9.360288835 podStartE2EDuration="20.86974874s" totalImagesPullingTime="11.484086537s" totalInitContainerRuntime="25.373368ms" isStatefulPod=false podCreationTimestamp="2026-07-26 15:19:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:20:02.867884334 +0000 UTC m=+87.157716454" watchObservedRunningTime="2026-07-26 15:20:02.86974874 +0000 UTC m=+87.159580858"
Jul 26 15:20:03.301613 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:03.301527    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:20:04.326187 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:04.326144    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:04.426747 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:04.426705    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:05.926607 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:05.926573    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:06.028186 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:06.028145    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:08.524030 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:08.523996    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:08.574547 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:08.574508    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:08.924277 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:08.924235    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:08.975538 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:08.975506    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:09.928284 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:09.928249    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:10.029390 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:10.029352    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:13.823724 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:13.823690    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:13.874384 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:13.874353    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:14.332756 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:14.332730    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:14.427046 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:14.427008    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:20.765151 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:20.765120    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gztc2"
Jul 26 15:20:21.340599 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:21.340563    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:20:21.345358 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:21.345333    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-775b87ff9b-hzhj9"
Jul 26 15:20:23.934541 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:23.934506    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:24.030503 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:24.030470    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:43.301278 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:43.301239    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:20:43.355895 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:43.355868    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:20:44.035552 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:44.035523    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:20:55.880842 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:55.880812    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:56.318654 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:56.318621    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:20:57.025933 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:57.025901    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 26 15:20:57.026355 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:57.026216    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="alertmanager" containerID="cri-o://9e487eb5b2c258ac09a68740d962a7a053a02dd85f9a0a63b27c9ad2c388974a" gracePeriod=120
Jul 26 15:20:57.026355 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:57.026283    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="kube-rbac-proxy-metric" containerID="cri-o://c1dd555713c53fe339125f768d32809bed3d95b920dcbd1138a1f658970134c0" gracePeriod=120
Jul 26 15:20:57.026355 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:57.026317    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="prom-label-proxy" containerID="cri-o://a96260a488d22c700aeb80a2ddd7943f136c1befa94fb698da25406eb510e8c0" gracePeriod=120
Jul 26 15:20:57.026529 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:57.026365    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="config-reloader" containerID="cri-o://4f78c2bb02ab9d9710d931a9434a5df1a2887c0c248d76cdd3e3e9d7bd78d5df" gracePeriod=120
Jul 26 15:20:57.026529 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:57.026315    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="kube-rbac-proxy-web" containerID="cri-o://465a95a09a61b6c76da7caa3b65b0de52c7b1ae94a2b1a95f048f8f12b834147" gracePeriod=120
Jul 26 15:20:57.026529 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:57.026389    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="kube-rbac-proxy" containerID="cri-o://15f04bace33bf09f83779b409a3cd51f9b6da4ab2c56c863f3eca2b1de631346" gracePeriod=120
Jul 26 15:20:58.076354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.076316    2559 generic.go:356] "Generic (PLEG): container finished" podID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerID="a96260a488d22c700aeb80a2ddd7943f136c1befa94fb698da25406eb510e8c0" exitCode=0
Jul 26 15:20:58.076354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.076342    2559 generic.go:356] "Generic (PLEG): container finished" podID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerID="15f04bace33bf09f83779b409a3cd51f9b6da4ab2c56c863f3eca2b1de631346" exitCode=0
Jul 26 15:20:58.076354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.076352    2559 generic.go:356] "Generic (PLEG): container finished" podID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerID="4f78c2bb02ab9d9710d931a9434a5df1a2887c0c248d76cdd3e3e9d7bd78d5df" exitCode=0
Jul 26 15:20:58.076354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.076359    2559 generic.go:356] "Generic (PLEG): container finished" podID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerID="9e487eb5b2c258ac09a68740d962a7a053a02dd85f9a0a63b27c9ad2c388974a" exitCode=0
Jul 26 15:20:58.076826 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.076385    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"113bfc00-208b-4a4f-9f4f-4775b2435f10","Type":"ContainerDied","Data":"a96260a488d22c700aeb80a2ddd7943f136c1befa94fb698da25406eb510e8c0"}
Jul 26 15:20:58.076826 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.076447    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"113bfc00-208b-4a4f-9f4f-4775b2435f10","Type":"ContainerDied","Data":"15f04bace33bf09f83779b409a3cd51f9b6da4ab2c56c863f3eca2b1de631346"}
Jul 26 15:20:58.076826 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.076479    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"113bfc00-208b-4a4f-9f4f-4775b2435f10","Type":"ContainerDied","Data":"4f78c2bb02ab9d9710d931a9434a5df1a2887c0c248d76cdd3e3e9d7bd78d5df"}
Jul 26 15:20:58.076826 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.076505    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"113bfc00-208b-4a4f-9f4f-4775b2435f10","Type":"ContainerDied","Data":"9e487eb5b2c258ac09a68740d962a7a053a02dd85f9a0a63b27c9ad2c388974a"}
Jul 26 15:20:58.267587 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.267563    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:58.332164 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.332080    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/113bfc00-208b-4a4f-9f4f-4775b2435f10-config-out\") pod \"113bfc00-208b-4a4f-9f4f-4775b2435f10\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") "
Jul 26 15:20:58.332164 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.332114    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-main-tls\") pod \"113bfc00-208b-4a4f-9f4f-4775b2435f10\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") "
Jul 26 15:20:58.332164 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.332141    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-cluster-tls-config\") pod \"113bfc00-208b-4a4f-9f4f-4775b2435f10\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") "
Jul 26 15:20:58.332164 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.332165    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dj46\" (UniqueName: \"kubernetes.io/projected/113bfc00-208b-4a4f-9f4f-4775b2435f10-kube-api-access-2dj46\") pod \"113bfc00-208b-4a4f-9f4f-4775b2435f10\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") "
Jul 26 15:20:58.332491 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.332190    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-kube-rbac-proxy-metric\") pod \"113bfc00-208b-4a4f-9f4f-4775b2435f10\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") "
Jul 26 15:20:58.332491 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.332239    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-kube-rbac-proxy\") pod \"113bfc00-208b-4a4f-9f4f-4775b2435f10\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") "
Jul 26 15:20:58.332491 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.332284    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/113bfc00-208b-4a4f-9f4f-4775b2435f10-alertmanager-main-db\") pod \"113bfc00-208b-4a4f-9f4f-4775b2435f10\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") "
Jul 26 15:20:58.332491 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.332324    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/113bfc00-208b-4a4f-9f4f-4775b2435f10-tls-assets\") pod \"113bfc00-208b-4a4f-9f4f-4775b2435f10\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") "
Jul 26 15:20:58.332491 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.332369    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/113bfc00-208b-4a4f-9f4f-4775b2435f10-metrics-client-ca\") pod \"113bfc00-208b-4a4f-9f4f-4775b2435f10\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") "
Jul 26 15:20:58.332491 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.332420    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-web-config\") pod \"113bfc00-208b-4a4f-9f4f-4775b2435f10\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") "
Jul 26 15:20:58.332491 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.332464    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/113bfc00-208b-4a4f-9f4f-4775b2435f10-alertmanager-trusted-ca-bundle\") pod \"113bfc00-208b-4a4f-9f4f-4775b2435f10\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") "
Jul 26 15:20:58.332834 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.332504    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-kube-rbac-proxy-web\") pod \"113bfc00-208b-4a4f-9f4f-4775b2435f10\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") "
Jul 26 15:20:58.332834 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.332542    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-config-volume\") pod \"113bfc00-208b-4a4f-9f4f-4775b2435f10\" (UID: \"113bfc00-208b-4a4f-9f4f-4775b2435f10\") "
Jul 26 15:20:58.333458 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.333426    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/113bfc00-208b-4a4f-9f4f-4775b2435f10-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "113bfc00-208b-4a4f-9f4f-4775b2435f10" (UID: "113bfc00-208b-4a4f-9f4f-4775b2435f10"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 15:20:58.333581 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.333500    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/113bfc00-208b-4a4f-9f4f-4775b2435f10-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "113bfc00-208b-4a4f-9f4f-4775b2435f10" (UID: "113bfc00-208b-4a4f-9f4f-4775b2435f10"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:20:58.333581 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.333563    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/113bfc00-208b-4a4f-9f4f-4775b2435f10-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "113bfc00-208b-4a4f-9f4f-4775b2435f10" (UID: "113bfc00-208b-4a4f-9f4f-4775b2435f10"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:20:58.335312 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.335249    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "113bfc00-208b-4a4f-9f4f-4775b2435f10" (UID: "113bfc00-208b-4a4f-9f4f-4775b2435f10"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:20:58.335450 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.335422    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/113bfc00-208b-4a4f-9f4f-4775b2435f10-config-out" (OuterVolumeSpecName: "config-out") pod "113bfc00-208b-4a4f-9f4f-4775b2435f10" (UID: "113bfc00-208b-4a4f-9f4f-4775b2435f10"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 15:20:58.335912 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.335809    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "113bfc00-208b-4a4f-9f4f-4775b2435f10" (UID: "113bfc00-208b-4a4f-9f4f-4775b2435f10"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:20:58.335912 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.335880    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/113bfc00-208b-4a4f-9f4f-4775b2435f10-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "113bfc00-208b-4a4f-9f4f-4775b2435f10" (UID: "113bfc00-208b-4a4f-9f4f-4775b2435f10"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:20:58.336810 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.336775    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "113bfc00-208b-4a4f-9f4f-4775b2435f10" (UID: "113bfc00-208b-4a4f-9f4f-4775b2435f10"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:20:58.336942 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.336864    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "113bfc00-208b-4a4f-9f4f-4775b2435f10" (UID: "113bfc00-208b-4a4f-9f4f-4775b2435f10"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:20:58.337145 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.337120    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-config-volume" (OuterVolumeSpecName: "config-volume") pod "113bfc00-208b-4a4f-9f4f-4775b2435f10" (UID: "113bfc00-208b-4a4f-9f4f-4775b2435f10"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:20:58.337223 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.337153    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/113bfc00-208b-4a4f-9f4f-4775b2435f10-kube-api-access-2dj46" (OuterVolumeSpecName: "kube-api-access-2dj46") pod "113bfc00-208b-4a4f-9f4f-4775b2435f10" (UID: "113bfc00-208b-4a4f-9f4f-4775b2435f10"). InnerVolumeSpecName "kube-api-access-2dj46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:20:58.340686 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.340639    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "113bfc00-208b-4a4f-9f4f-4775b2435f10" (UID: "113bfc00-208b-4a4f-9f4f-4775b2435f10"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:20:58.347502 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.347473    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-web-config" (OuterVolumeSpecName: "web-config") pod "113bfc00-208b-4a4f-9f4f-4775b2435f10" (UID: "113bfc00-208b-4a4f-9f4f-4775b2435f10"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:20:58.433516 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.433483    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:20:58.433516 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.433511    2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/113bfc00-208b-4a4f-9f4f-4775b2435f10-alertmanager-main-db\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:20:58.433516 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.433522    2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/113bfc00-208b-4a4f-9f4f-4775b2435f10-tls-assets\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:20:58.433744 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.433531    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/113bfc00-208b-4a4f-9f4f-4775b2435f10-metrics-client-ca\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:20:58.433744 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.433559    2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-web-config\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:20:58.433744 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.433570    2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/113bfc00-208b-4a4f-9f4f-4775b2435f10-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:20:58.433744 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.433579    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:20:58.433744 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.433589    2559 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:20:58.433744 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.433597    2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/113bfc00-208b-4a4f-9f4f-4775b2435f10-config-out\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:20:58.433744 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.433604    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-main-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:20:58.433744 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.433612    2559 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-cluster-tls-config\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:20:58.433744 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.433620    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dj46\" (UniqueName: \"kubernetes.io/projected/113bfc00-208b-4a4f-9f4f-4775b2435f10-kube-api-access-2dj46\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:20:58.433744 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:58.433628    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/113bfc00-208b-4a4f-9f4f-4775b2435f10-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:20:59.083353 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.083311    2559 generic.go:356] "Generic (PLEG): container finished" podID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerID="c1dd555713c53fe339125f768d32809bed3d95b920dcbd1138a1f658970134c0" exitCode=0
Jul 26 15:20:59.083353 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.083343    2559 generic.go:356] "Generic (PLEG): container finished" podID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerID="465a95a09a61b6c76da7caa3b65b0de52c7b1ae94a2b1a95f048f8f12b834147" exitCode=0
Jul 26 15:20:59.083810 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.083390    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"113bfc00-208b-4a4f-9f4f-4775b2435f10","Type":"ContainerDied","Data":"c1dd555713c53fe339125f768d32809bed3d95b920dcbd1138a1f658970134c0"}
Jul 26 15:20:59.083810 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.083472    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"113bfc00-208b-4a4f-9f4f-4775b2435f10","Type":"ContainerDied","Data":"465a95a09a61b6c76da7caa3b65b0de52c7b1ae94a2b1a95f048f8f12b834147"}
Jul 26 15:20:59.083810 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.083402    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.083810 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.083510    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"113bfc00-208b-4a4f-9f4f-4775b2435f10","Type":"ContainerDied","Data":"18067642bdb9413ec84cd90bfccb7c4b37af269f2d36b5ee383cf2a62cbafdf0"}
Jul 26 15:20:59.083810 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.083543    2559 scope.go:117] "RemoveContainer" containerID="a96260a488d22c700aeb80a2ddd7943f136c1befa94fb698da25406eb510e8c0"
Jul 26 15:20:59.091672 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.091547    2559 scope.go:117] "RemoveContainer" containerID="c1dd555713c53fe339125f768d32809bed3d95b920dcbd1138a1f658970134c0"
Jul 26 15:20:59.106439 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.106413    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 26 15:20:59.106675 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.106659    2559 scope.go:117] "RemoveContainer" containerID="15f04bace33bf09f83779b409a3cd51f9b6da4ab2c56c863f3eca2b1de631346"
Jul 26 15:20:59.112537 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.112513    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 26 15:20:59.114521 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.114503    2559 scope.go:117] "RemoveContainer" containerID="465a95a09a61b6c76da7caa3b65b0de52c7b1ae94a2b1a95f048f8f12b834147"
Jul 26 15:20:59.121344 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.121326    2559 scope.go:117] "RemoveContainer" containerID="4f78c2bb02ab9d9710d931a9434a5df1a2887c0c248d76cdd3e3e9d7bd78d5df"
Jul 26 15:20:59.128389 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.128361    2559 scope.go:117] "RemoveContainer" containerID="9e487eb5b2c258ac09a68740d962a7a053a02dd85f9a0a63b27c9ad2c388974a"
Jul 26 15:20:59.135713 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.135695    2559 scope.go:117] "RemoveContainer" containerID="a0419517f74b9ff726e51e781bfa65a31518ad5cbba77658ffe50c77c4647273"
Jul 26 15:20:59.138768 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.138743    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 26 15:20:59.139069 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139054    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="kube-rbac-proxy"
Jul 26 15:20:59.139069 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139069    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="kube-rbac-proxy"
Jul 26 15:20:59.139197 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139082    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="prom-label-proxy"
Jul 26 15:20:59.139197 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139087    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="prom-label-proxy"
Jul 26 15:20:59.139197 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139096    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="alertmanager"
Jul 26 15:20:59.139197 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139101    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="alertmanager"
Jul 26 15:20:59.139197 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139109    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="init-config-reloader"
Jul 26 15:20:59.139197 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139115    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="init-config-reloader"
Jul 26 15:20:59.139197 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139130    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="kube-rbac-proxy-web"
Jul 26 15:20:59.139197 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139135    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="kube-rbac-proxy-web"
Jul 26 15:20:59.139197 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139144    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="config-reloader"
Jul 26 15:20:59.139197 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139149    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="config-reloader"
Jul 26 15:20:59.139197 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139155    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="kube-rbac-proxy-metric"
Jul 26 15:20:59.139197 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139160    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="kube-rbac-proxy-metric"
Jul 26 15:20:59.139197 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139220    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="alertmanager"
Jul 26 15:20:59.139197 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139231    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="kube-rbac-proxy-web"
Jul 26 15:20:59.139670 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139236    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="config-reloader"
Jul 26 15:20:59.139670 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139242    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="kube-rbac-proxy"
Jul 26 15:20:59.139670 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139249    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="kube-rbac-proxy-metric"
Jul 26 15:20:59.139670 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.139254    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" containerName="prom-label-proxy"
Jul 26 15:20:59.142840 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.142814    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.144097 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.143931    2559 scope.go:117] "RemoveContainer" containerID="a96260a488d22c700aeb80a2ddd7943f136c1befa94fb698da25406eb510e8c0"
Jul 26 15:20:59.144256 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:20:59.144237    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a96260a488d22c700aeb80a2ddd7943f136c1befa94fb698da25406eb510e8c0\": container with ID starting with a96260a488d22c700aeb80a2ddd7943f136c1befa94fb698da25406eb510e8c0 not found: ID does not exist" containerID="a96260a488d22c700aeb80a2ddd7943f136c1befa94fb698da25406eb510e8c0"
Jul 26 15:20:59.144306 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.144264    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a96260a488d22c700aeb80a2ddd7943f136c1befa94fb698da25406eb510e8c0"} err="failed to get container status \"a96260a488d22c700aeb80a2ddd7943f136c1befa94fb698da25406eb510e8c0\": rpc error: code = NotFound desc = could not find container \"a96260a488d22c700aeb80a2ddd7943f136c1befa94fb698da25406eb510e8c0\": container with ID starting with a96260a488d22c700aeb80a2ddd7943f136c1befa94fb698da25406eb510e8c0 not found: ID does not exist"
Jul 26 15:20:59.144306 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.144294    2559 scope.go:117] "RemoveContainer" containerID="c1dd555713c53fe339125f768d32809bed3d95b920dcbd1138a1f658970134c0"
Jul 26 15:20:59.144529 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:20:59.144515    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1dd555713c53fe339125f768d32809bed3d95b920dcbd1138a1f658970134c0\": container with ID starting with c1dd555713c53fe339125f768d32809bed3d95b920dcbd1138a1f658970134c0 not found: ID does not exist" containerID="c1dd555713c53fe339125f768d32809bed3d95b920dcbd1138a1f658970134c0"
Jul 26 15:20:59.144569 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.144532    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1dd555713c53fe339125f768d32809bed3d95b920dcbd1138a1f658970134c0"} err="failed to get container status \"c1dd555713c53fe339125f768d32809bed3d95b920dcbd1138a1f658970134c0\": rpc error: code = NotFound desc = could not find container \"c1dd555713c53fe339125f768d32809bed3d95b920dcbd1138a1f658970134c0\": container with ID starting with c1dd555713c53fe339125f768d32809bed3d95b920dcbd1138a1f658970134c0 not found: ID does not exist"
Jul 26 15:20:59.144569 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.144549    2559 scope.go:117] "RemoveContainer" containerID="15f04bace33bf09f83779b409a3cd51f9b6da4ab2c56c863f3eca2b1de631346"
Jul 26 15:20:59.144791 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:20:59.144766    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15f04bace33bf09f83779b409a3cd51f9b6da4ab2c56c863f3eca2b1de631346\": container with ID starting with 15f04bace33bf09f83779b409a3cd51f9b6da4ab2c56c863f3eca2b1de631346 not found: ID does not exist" containerID="15f04bace33bf09f83779b409a3cd51f9b6da4ab2c56c863f3eca2b1de631346"
Jul 26 15:20:59.144887 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.144792    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15f04bace33bf09f83779b409a3cd51f9b6da4ab2c56c863f3eca2b1de631346"} err="failed to get container status \"15f04bace33bf09f83779b409a3cd51f9b6da4ab2c56c863f3eca2b1de631346\": rpc error: code = NotFound desc = could not find container \"15f04bace33bf09f83779b409a3cd51f9b6da4ab2c56c863f3eca2b1de631346\": container with ID starting with 15f04bace33bf09f83779b409a3cd51f9b6da4ab2c56c863f3eca2b1de631346 not found: ID does not exist"
Jul 26 15:20:59.144887 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.144811    2559 scope.go:117] "RemoveContainer" containerID="465a95a09a61b6c76da7caa3b65b0de52c7b1ae94a2b1a95f048f8f12b834147"
Jul 26 15:20:59.144998 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.144964    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-c9nlk\""
Jul 26 15:20:59.144998 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.144963    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 26 15:20:59.145134 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:20:59.145112    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"465a95a09a61b6c76da7caa3b65b0de52c7b1ae94a2b1a95f048f8f12b834147\": container with ID starting with 465a95a09a61b6c76da7caa3b65b0de52c7b1ae94a2b1a95f048f8f12b834147 not found: ID does not exist" containerID="465a95a09a61b6c76da7caa3b65b0de52c7b1ae94a2b1a95f048f8f12b834147"
Jul 26 15:20:59.145189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.145143    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"465a95a09a61b6c76da7caa3b65b0de52c7b1ae94a2b1a95f048f8f12b834147"} err="failed to get container status \"465a95a09a61b6c76da7caa3b65b0de52c7b1ae94a2b1a95f048f8f12b834147\": rpc error: code = NotFound desc = could not find container \"465a95a09a61b6c76da7caa3b65b0de52c7b1ae94a2b1a95f048f8f12b834147\": container with ID starting with 465a95a09a61b6c76da7caa3b65b0de52c7b1ae94a2b1a95f048f8f12b834147 not found: ID does not exist"
Jul 26 15:20:59.145189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.145163    2559 scope.go:117] "RemoveContainer" containerID="4f78c2bb02ab9d9710d931a9434a5df1a2887c0c248d76cdd3e3e9d7bd78d5df"
Jul 26 15:20:59.145189 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.145175    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 26 15:20:59.145374 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.145188    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 26 15:20:59.145425 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.145407    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 26 15:20:59.145425 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.145420    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 26 15:20:59.145515 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.145427    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 26 15:20:59.145515 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.145442    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 26 15:20:59.145515 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.145490    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 26 15:20:59.145749 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:20:59.145730    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f78c2bb02ab9d9710d931a9434a5df1a2887c0c248d76cdd3e3e9d7bd78d5df\": container with ID starting with 4f78c2bb02ab9d9710d931a9434a5df1a2887c0c248d76cdd3e3e9d7bd78d5df not found: ID does not exist" containerID="4f78c2bb02ab9d9710d931a9434a5df1a2887c0c248d76cdd3e3e9d7bd78d5df"
Jul 26 15:20:59.145807 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.145760    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f78c2bb02ab9d9710d931a9434a5df1a2887c0c248d76cdd3e3e9d7bd78d5df"} err="failed to get container status \"4f78c2bb02ab9d9710d931a9434a5df1a2887c0c248d76cdd3e3e9d7bd78d5df\": rpc error: code = NotFound desc = could not find container \"4f78c2bb02ab9d9710d931a9434a5df1a2887c0c248d76cdd3e3e9d7bd78d5df\": container with ID starting with 4f78c2bb02ab9d9710d931a9434a5df1a2887c0c248d76cdd3e3e9d7bd78d5df not found: ID does not exist"
Jul 26 15:20:59.145807 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.145782    2559 scope.go:117] "RemoveContainer" containerID="9e487eb5b2c258ac09a68740d962a7a053a02dd85f9a0a63b27c9ad2c388974a"
Jul 26 15:20:59.146106 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:20:59.146078    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e487eb5b2c258ac09a68740d962a7a053a02dd85f9a0a63b27c9ad2c388974a\": container with ID starting with 9e487eb5b2c258ac09a68740d962a7a053a02dd85f9a0a63b27c9ad2c388974a not found: ID does not exist" containerID="9e487eb5b2c258ac09a68740d962a7a053a02dd85f9a0a63b27c9ad2c388974a"
Jul 26 15:20:59.146241 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.146110    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e487eb5b2c258ac09a68740d962a7a053a02dd85f9a0a63b27c9ad2c388974a"} err="failed to get container status \"9e487eb5b2c258ac09a68740d962a7a053a02dd85f9a0a63b27c9ad2c388974a\": rpc error: code = NotFound desc = could not find container \"9e487eb5b2c258ac09a68740d962a7a053a02dd85f9a0a63b27c9ad2c388974a\": container with ID starting with 9e487eb5b2c258ac09a68740d962a7a053a02dd85f9a0a63b27c9ad2c388974a not found: ID does not exist"
Jul 26 15:20:59.146241 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.146134    2559 scope.go:117] "RemoveContainer" containerID="a0419517f74b9ff726e51e781bfa65a31518ad5cbba77658ffe50c77c4647273"
Jul 26 15:20:59.146397 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:20:59.146382    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0419517f74b9ff726e51e781bfa65a31518ad5cbba77658ffe50c77c4647273\": container with ID starting with a0419517f74b9ff726e51e781bfa65a31518ad5cbba77658ffe50c77c4647273 not found: ID does not exist" containerID="a0419517f74b9ff726e51e781bfa65a31518ad5cbba77658ffe50c77c4647273"
Jul 26 15:20:59.146456 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.146404    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0419517f74b9ff726e51e781bfa65a31518ad5cbba77658ffe50c77c4647273"} err="failed to get container status \"a0419517f74b9ff726e51e781bfa65a31518ad5cbba77658ffe50c77c4647273\": rpc error: code = NotFound desc = could not find container \"a0419517f74b9ff726e51e781bfa65a31518ad5cbba77658ffe50c77c4647273\": container with ID starting with a0419517f74b9ff726e51e781bfa65a31518ad5cbba77658ffe50c77c4647273 not found: ID does not exist"
Jul 26 15:20:59.146456 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.146417    2559 scope.go:117] "RemoveContainer" containerID="a96260a488d22c700aeb80a2ddd7943f136c1befa94fb698da25406eb510e8c0"
Jul 26 15:20:59.146634 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.146612    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a96260a488d22c700aeb80a2ddd7943f136c1befa94fb698da25406eb510e8c0"} err="failed to get container status \"a96260a488d22c700aeb80a2ddd7943f136c1befa94fb698da25406eb510e8c0\": rpc error: code = NotFound desc = could not find container \"a96260a488d22c700aeb80a2ddd7943f136c1befa94fb698da25406eb510e8c0\": container with ID starting with a96260a488d22c700aeb80a2ddd7943f136c1befa94fb698da25406eb510e8c0 not found: ID does not exist"
Jul 26 15:20:59.146634 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.146633    2559 scope.go:117] "RemoveContainer" containerID="c1dd555713c53fe339125f768d32809bed3d95b920dcbd1138a1f658970134c0"
Jul 26 15:20:59.146886 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.146840    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1dd555713c53fe339125f768d32809bed3d95b920dcbd1138a1f658970134c0"} err="failed to get container status \"c1dd555713c53fe339125f768d32809bed3d95b920dcbd1138a1f658970134c0\": rpc error: code = NotFound desc = could not find container \"c1dd555713c53fe339125f768d32809bed3d95b920dcbd1138a1f658970134c0\": container with ID starting with c1dd555713c53fe339125f768d32809bed3d95b920dcbd1138a1f658970134c0 not found: ID does not exist"
Jul 26 15:20:59.146886 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.146860    2559 scope.go:117] "RemoveContainer" containerID="15f04bace33bf09f83779b409a3cd51f9b6da4ab2c56c863f3eca2b1de631346"
Jul 26 15:20:59.147177 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.147136    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15f04bace33bf09f83779b409a3cd51f9b6da4ab2c56c863f3eca2b1de631346"} err="failed to get container status \"15f04bace33bf09f83779b409a3cd51f9b6da4ab2c56c863f3eca2b1de631346\": rpc error: code = NotFound desc = could not find container \"15f04bace33bf09f83779b409a3cd51f9b6da4ab2c56c863f3eca2b1de631346\": container with ID starting with 15f04bace33bf09f83779b409a3cd51f9b6da4ab2c56c863f3eca2b1de631346 not found: ID does not exist"
Jul 26 15:20:59.147177 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.147162    2559 scope.go:117] "RemoveContainer" containerID="465a95a09a61b6c76da7caa3b65b0de52c7b1ae94a2b1a95f048f8f12b834147"
Jul 26 15:20:59.147560 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.147465    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"465a95a09a61b6c76da7caa3b65b0de52c7b1ae94a2b1a95f048f8f12b834147"} err="failed to get container status \"465a95a09a61b6c76da7caa3b65b0de52c7b1ae94a2b1a95f048f8f12b834147\": rpc error: code = NotFound desc = could not find container \"465a95a09a61b6c76da7caa3b65b0de52c7b1ae94a2b1a95f048f8f12b834147\": container with ID starting with 465a95a09a61b6c76da7caa3b65b0de52c7b1ae94a2b1a95f048f8f12b834147 not found: ID does not exist"
Jul 26 15:20:59.147560 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.147495    2559 scope.go:117] "RemoveContainer" containerID="4f78c2bb02ab9d9710d931a9434a5df1a2887c0c248d76cdd3e3e9d7bd78d5df"
Jul 26 15:20:59.147889 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.147861    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f78c2bb02ab9d9710d931a9434a5df1a2887c0c248d76cdd3e3e9d7bd78d5df"} err="failed to get container status \"4f78c2bb02ab9d9710d931a9434a5df1a2887c0c248d76cdd3e3e9d7bd78d5df\": rpc error: code = NotFound desc = could not find container \"4f78c2bb02ab9d9710d931a9434a5df1a2887c0c248d76cdd3e3e9d7bd78d5df\": container with ID starting with 4f78c2bb02ab9d9710d931a9434a5df1a2887c0c248d76cdd3e3e9d7bd78d5df not found: ID does not exist"
Jul 26 15:20:59.147889 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.147887    2559 scope.go:117] "RemoveContainer" containerID="9e487eb5b2c258ac09a68740d962a7a053a02dd85f9a0a63b27c9ad2c388974a"
Jul 26 15:20:59.148132 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.148116    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e487eb5b2c258ac09a68740d962a7a053a02dd85f9a0a63b27c9ad2c388974a"} err="failed to get container status \"9e487eb5b2c258ac09a68740d962a7a053a02dd85f9a0a63b27c9ad2c388974a\": rpc error: code = NotFound desc = could not find container \"9e487eb5b2c258ac09a68740d962a7a053a02dd85f9a0a63b27c9ad2c388974a\": container with ID starting with 9e487eb5b2c258ac09a68740d962a7a053a02dd85f9a0a63b27c9ad2c388974a not found: ID does not exist"
Jul 26 15:20:59.148196 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.148133    2559 scope.go:117] "RemoveContainer" containerID="a0419517f74b9ff726e51e781bfa65a31518ad5cbba77658ffe50c77c4647273"
Jul 26 15:20:59.148417 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.148389    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0419517f74b9ff726e51e781bfa65a31518ad5cbba77658ffe50c77c4647273"} err="failed to get container status \"a0419517f74b9ff726e51e781bfa65a31518ad5cbba77658ffe50c77c4647273\": rpc error: code = NotFound desc = could not find container \"a0419517f74b9ff726e51e781bfa65a31518ad5cbba77658ffe50c77c4647273\": container with ID starting with a0419517f74b9ff726e51e781bfa65a31518ad5cbba77658ffe50c77c4647273 not found: ID does not exist"
Jul 26 15:20:59.153532 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.152453    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 26 15:20:59.157069 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.157049    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 26 15:20:59.241250 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.241185    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.241250 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.241256    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.241490 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.241287    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-config-volume\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.241490 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.241310    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.241490 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.241382    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19d25697-6503-4a21-ac72-21388c1eda33-config-out\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.241490 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.241421    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d25697-6503-4a21-ac72-21388c1eda33-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.241490 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.241470    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19d25697-6503-4a21-ac72-21388c1eda33-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.241685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.241510    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19d25697-6503-4a21-ac72-21388c1eda33-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.241685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.241531    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-web-config\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.241685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.241551    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.241685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.241591    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.241685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.241616    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19d25697-6503-4a21-ac72-21388c1eda33-tls-assets\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.241685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.241662    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-975z6\" (UniqueName: \"kubernetes.io/projected/19d25697-6503-4a21-ac72-21388c1eda33-kube-api-access-975z6\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.343100 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.343005    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19d25697-6503-4a21-ac72-21388c1eda33-tls-assets\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.343100 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.343047    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-975z6\" (UniqueName: \"kubernetes.io/projected/19d25697-6503-4a21-ac72-21388c1eda33-kube-api-access-975z6\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.343100 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.343077    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.343100 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.343093    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.343441 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.343243    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-config-volume\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.343441 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.343299    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.343441 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.343361    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19d25697-6503-4a21-ac72-21388c1eda33-config-out\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.343441 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.343392    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d25697-6503-4a21-ac72-21388c1eda33-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.343441 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.343425    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19d25697-6503-4a21-ac72-21388c1eda33-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.343691 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.343479    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19d25697-6503-4a21-ac72-21388c1eda33-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.343691 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.343523    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-web-config\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.343691 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.343559    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.343691 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.343596    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.344586 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.344552    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d25697-6503-4a21-ac72-21388c1eda33-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.344848 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.344823    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19d25697-6503-4a21-ac72-21388c1eda33-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.345462 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.345437    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19d25697-6503-4a21-ac72-21388c1eda33-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.345890 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.345864    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19d25697-6503-4a21-ac72-21388c1eda33-config-out\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.346081 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.346044    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.346246 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.346192    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.346519 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.346494    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.346774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.346755    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19d25697-6503-4a21-ac72-21388c1eda33-tls-assets\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.346894 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.346829    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-config-volume\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.346992 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.346969    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.347802 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.347776    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-web-config\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.348040 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.348021    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19d25697-6503-4a21-ac72-21388c1eda33-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.351063 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.351045    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-975z6\" (UniqueName: \"kubernetes.io/projected/19d25697-6503-4a21-ac72-21388c1eda33-kube-api-access-975z6\") pod \"alertmanager-main-0\" (UID: \"19d25697-6503-4a21-ac72-21388c1eda33\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.459028 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.458993    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:20:59.609078 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:20:59.608983    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 26 15:20:59.613789 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:20:59.613761    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19d25697_6503_4a21_ac72_21388c1eda33.slice/crio-ffb2d7f386d2a97f8057b540197fa94db142b998b0cb4d616ab841334cdefaad WatchSource:0}: Error finding container ffb2d7f386d2a97f8057b540197fa94db142b998b0cb4d616ab841334cdefaad: Status 404 returned error can't find the container with id ffb2d7f386d2a97f8057b540197fa94db142b998b0cb4d616ab841334cdefaad
Jul 26 15:21:00.088408 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:00.088371    2559 generic.go:356] "Generic (PLEG): container finished" podID="19d25697-6503-4a21-ac72-21388c1eda33" containerID="c34ea342f3362c4b2f3f6e63bbd3598f9f9e5952024314e1275ee41109c2f35e" exitCode=0
Jul 26 15:21:00.088819 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:00.088454    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d25697-6503-4a21-ac72-21388c1eda33","Type":"ContainerDied","Data":"c34ea342f3362c4b2f3f6e63bbd3598f9f9e5952024314e1275ee41109c2f35e"}
Jul 26 15:21:00.088819 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:00.088518    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d25697-6503-4a21-ac72-21388c1eda33","Type":"ContainerStarted","Data":"ffb2d7f386d2a97f8057b540197fa94db142b998b0cb4d616ab841334cdefaad"}
Jul 26 15:21:00.193084 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:00.193044    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="113bfc00-208b-4a4f-9f4f-4775b2435f10" path="/var/lib/kubelet/pods/113bfc00-208b-4a4f-9f4f-4775b2435f10/volumes"
Jul 26 15:21:01.097917 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.097885    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d25697-6503-4a21-ac72-21388c1eda33","Type":"ContainerStarted","Data":"59b4e112eb5ba52eda57171710834099e61c47ad8ba72124b99bedd337fdd0d8"}
Jul 26 15:21:01.098370 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.097945    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d25697-6503-4a21-ac72-21388c1eda33","Type":"ContainerStarted","Data":"371383ac816b8e07b5cecaa4e0b739cccb9ef7103bcad85eae92f12d778d9a10"}
Jul 26 15:21:01.098370 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.097972    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d25697-6503-4a21-ac72-21388c1eda33","Type":"ContainerStarted","Data":"ab7bef845aba63a565e98bfbc38d2e75f2eda6810041024f0e52e23de69e8162"}
Jul 26 15:21:01.098370 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.097999    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d25697-6503-4a21-ac72-21388c1eda33","Type":"ContainerStarted","Data":"fab9d7ce6396b36c23e5a51fc616857522aa9115a2e4c39f527a51ebb3bfd3ff"}
Jul 26 15:21:01.098370 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.098023    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d25697-6503-4a21-ac72-21388c1eda33","Type":"ContainerStarted","Data":"f662895823a66d56bcde0d668a1e855a2f4622fdf70d5ef8ab58632c839df66e"}
Jul 26 15:21:01.098370 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.098045    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d25697-6503-4a21-ac72-21388c1eda33","Type":"ContainerStarted","Data":"76b857188a796cfff575f980f7687d65caf9b9a3033f5aa0c1318a810802c9ce"}
Jul 26 15:21:01.128691 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.128623    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.112977104 podStartE2EDuration="2.128601323s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.624219ms" isStatefulPod=false podCreationTimestamp="2026-07-26 15:20:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:21:01.127697057 +0000 UTC m=+145.417529199" watchObservedRunningTime="2026-07-26 15:21:01.128601323 +0000 UTC m=+145.418433443"
Jul 26 15:21:01.316630 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.316594    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 26 15:21:01.316989 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.316950    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="kube-rbac-proxy-web" containerID="cri-o://e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d" gracePeriod=600
Jul 26 15:21:01.317133 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.317018    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="prometheus" containerID="cri-o://d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b" gracePeriod=600
Jul 26 15:21:01.317133 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.316950    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="kube-rbac-proxy" containerID="cri-o://81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1" gracePeriod=600
Jul 26 15:21:01.317133 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.316989    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="kube-rbac-proxy-thanos" containerID="cri-o://867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9" gracePeriod=600
Jul 26 15:21:01.317326 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.316969    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="thanos-sidecar" containerID="cri-o://f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421" gracePeriod=600
Jul 26 15:21:01.317326 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.316969    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="config-reloader" containerID="cri-o://e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2" gracePeriod=600
Jul 26 15:21:01.570601 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.570575    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:01.664664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.664580    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.664664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.664631    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-grpc-tls\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.664664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.664657    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.664937 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.664686    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-web-config\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.664937 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.664750    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc5bea94-f396-4d67-ae50-b070a870a64e-config-out\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.664937 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.664832    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-rulefiles-0\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.664937 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.664894    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-rulefiles-1\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.664937 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.664926    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-configmap-kubelet-serving-ca-bundle\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.665194 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.664953    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-trusted-ca-bundle\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.665194 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.664989    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-thanos-prometheus-http-client-file\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.665194 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.665015    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-config\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.665194 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.665039    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-rulefiles-2\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.665194 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.665093    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-configmap-metrics-client-ca\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.665194 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.665128    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnc9q\" (UniqueName: \"kubernetes.io/projected/bc5bea94-f396-4d67-ae50-b070a870a64e-kube-api-access-wnc9q\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.665194 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.665153    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-metrics-client-certs\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.665194 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.665191    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-kube-rbac-proxy\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.665605 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.665241    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-prometheus-k8s-tls\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.665605 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.665276    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc5bea94-f396-4d67-ae50-b070a870a64e-tls-assets\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.665605 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.665307    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-configmap-serving-certs-ca-bundle\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.665605 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.665333    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-db\") pod \"bc5bea94-f396-4d67-ae50-b070a870a64e\" (UID: \"bc5bea94-f396-4d67-ae50-b070a870a64e\") "
Jul 26 15:21:01.665980 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.665948    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:21:01.666078 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.666043    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:21:01.666456 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.666413    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:21:01.666733 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.666711    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:21:01.666807 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.666712    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 15:21:01.667600 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.667564    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:21:01.667979 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.667871    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:21:01.668109 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.668081    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:21:01.668180 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.668114    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:21:01.668464 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.668335    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:21:01.668716 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.668674    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:21:01.668802 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.668770    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5bea94-f396-4d67-ae50-b070a870a64e-config-out" (OuterVolumeSpecName: "config-out") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 15:21:01.668802 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.668777    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:21:01.669389 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.669358    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:21:01.669726 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.669703    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:21:01.669971 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.669945    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-config" (OuterVolumeSpecName: "config") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:21:01.670076 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.670054    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc5bea94-f396-4d67-ae50-b070a870a64e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:21:01.670545 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.670515    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:21:01.670642 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.670599    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc5bea94-f396-4d67-ae50-b070a870a64e-kube-api-access-wnc9q" (OuterVolumeSpecName: "kube-api-access-wnc9q") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "kube-api-access-wnc9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:21:01.679753 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.679725    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-web-config" (OuterVolumeSpecName: "web-config") pod "bc5bea94-f396-4d67-ae50-b070a870a64e" (UID: "bc5bea94-f396-4d67-ae50-b070a870a64e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:21:01.766277 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766235    2559 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-grpc-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766277 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766271    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766277 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766282    2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-web-config\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766292    2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc5bea94-f396-4d67-ae50-b070a870a64e-config-out\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766301    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766310    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766319    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766327    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766336    2559 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766344    2559 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-config\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766351    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766360    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-configmap-metrics-client-ca\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766368    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnc9q\" (UniqueName: \"kubernetes.io/projected/bc5bea94-f396-4d67-ae50-b070a870a64e-kube-api-access-wnc9q\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766376    2559 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-metrics-client-certs\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766384    2559 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-kube-rbac-proxy\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766393    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766401    2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc5bea94-f396-4d67-ae50-b070a870a64e-tls-assets\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766408    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc5bea94-f396-4d67-ae50-b070a870a64e-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766418    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bc5bea94-f396-4d67-ae50-b070a870a64e-prometheus-k8s-db\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:01.766508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:01.766426    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bc5bea94-f396-4d67-ae50-b070a870a64e-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:21:02.111262 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.111228    2559 generic.go:356] "Generic (PLEG): container finished" podID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerID="867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9" exitCode=0
Jul 26 15:21:02.111262 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.111259    2559 generic.go:356] "Generic (PLEG): container finished" podID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerID="81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1" exitCode=0
Jul 26 15:21:02.111262 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.111267    2559 generic.go:356] "Generic (PLEG): container finished" podID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerID="e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d" exitCode=0
Jul 26 15:21:02.111730 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.111275    2559 generic.go:356] "Generic (PLEG): container finished" podID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerID="f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421" exitCode=0
Jul 26 15:21:02.111730 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.111283    2559 generic.go:356] "Generic (PLEG): container finished" podID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerID="e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2" exitCode=0
Jul 26 15:21:02.111730 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.111291    2559 generic.go:356] "Generic (PLEG): container finished" podID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerID="d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b" exitCode=0
Jul 26 15:21:02.111730 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.111310    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc5bea94-f396-4d67-ae50-b070a870a64e","Type":"ContainerDied","Data":"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9"}
Jul 26 15:21:02.111730 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.111329    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.111730 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.111381    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc5bea94-f396-4d67-ae50-b070a870a64e","Type":"ContainerDied","Data":"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1"}
Jul 26 15:21:02.111730 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.111408    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc5bea94-f396-4d67-ae50-b070a870a64e","Type":"ContainerDied","Data":"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d"}
Jul 26 15:21:02.111730 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.111433    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc5bea94-f396-4d67-ae50-b070a870a64e","Type":"ContainerDied","Data":"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421"}
Jul 26 15:21:02.111730 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.111457    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc5bea94-f396-4d67-ae50-b070a870a64e","Type":"ContainerDied","Data":"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2"}
Jul 26 15:21:02.111730 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.111473    2559 scope.go:117] "RemoveContainer" containerID="867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9"
Jul 26 15:21:02.111730 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.111480    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc5bea94-f396-4d67-ae50-b070a870a64e","Type":"ContainerDied","Data":"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b"}
Jul 26 15:21:02.111730 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.111514    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc5bea94-f396-4d67-ae50-b070a870a64e","Type":"ContainerDied","Data":"cfd9d6fb61850f58fc4a961331d450d28a0cce110bbda60de427d4fc7e4f00e6"}
Jul 26 15:21:02.120468 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.120445    2559 scope.go:117] "RemoveContainer" containerID="81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1"
Jul 26 15:21:02.127732 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.127711    2559 scope.go:117] "RemoveContainer" containerID="e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d"
Jul 26 15:21:02.134663 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.134635    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 26 15:21:02.135393 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.135377    2559 scope.go:117] "RemoveContainer" containerID="f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421"
Jul 26 15:21:02.138530 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.138508    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 26 15:21:02.142921 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.142903    2559 scope.go:117] "RemoveContainer" containerID="e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2"
Jul 26 15:21:02.149813 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.149795    2559 scope.go:117] "RemoveContainer" containerID="d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b"
Jul 26 15:21:02.157629 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.157611    2559 scope.go:117] "RemoveContainer" containerID="31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c"
Jul 26 15:21:02.161622 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.161598    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 26 15:21:02.161952 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.161931    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="init-config-reloader"
Jul 26 15:21:02.162045 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.161954    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="init-config-reloader"
Jul 26 15:21:02.162045 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.161966    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="config-reloader"
Jul 26 15:21:02.162045 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.161975    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="config-reloader"
Jul 26 15:21:02.162045 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.161994    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="prometheus"
Jul 26 15:21:02.162045 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.162002    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="prometheus"
Jul 26 15:21:02.162045 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.162012    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="kube-rbac-proxy-web"
Jul 26 15:21:02.162045 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.162018    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="kube-rbac-proxy-web"
Jul 26 15:21:02.162045 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.162025    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="kube-rbac-proxy-thanos"
Jul 26 15:21:02.162045 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.162031    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="kube-rbac-proxy-thanos"
Jul 26 15:21:02.162045 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.162041    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="thanos-sidecar"
Jul 26 15:21:02.162045 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.162046    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="thanos-sidecar"
Jul 26 15:21:02.162411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.162057    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="kube-rbac-proxy"
Jul 26 15:21:02.162411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.162062    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="kube-rbac-proxy"
Jul 26 15:21:02.162411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.162131    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="kube-rbac-proxy"
Jul 26 15:21:02.162411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.162141    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="thanos-sidecar"
Jul 26 15:21:02.162411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.162148    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="kube-rbac-proxy-thanos"
Jul 26 15:21:02.162411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.162160    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="kube-rbac-proxy-web"
Jul 26 15:21:02.162411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.162169    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="config-reloader"
Jul 26 15:21:02.162411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.162177    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" containerName="prometheus"
Jul 26 15:21:02.165852 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.165834    2559 scope.go:117] "RemoveContainer" containerID="867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9"
Jul 26 15:21:02.166191 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:21:02.166171    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9\": container with ID starting with 867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9 not found: ID does not exist" containerID="867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9"
Jul 26 15:21:02.166279 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.166219    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9"} err="failed to get container status \"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9\": rpc error: code = NotFound desc = could not find container \"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9\": container with ID starting with 867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9 not found: ID does not exist"
Jul 26 15:21:02.166279 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.166240    2559 scope.go:117] "RemoveContainer" containerID="81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1"
Jul 26 15:21:02.166488 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:21:02.166472    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1\": container with ID starting with 81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1 not found: ID does not exist" containerID="81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1"
Jul 26 15:21:02.166529 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.166492    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1"} err="failed to get container status \"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1\": rpc error: code = NotFound desc = could not find container \"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1\": container with ID starting with 81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1 not found: ID does not exist"
Jul 26 15:21:02.166529 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.166508    2559 scope.go:117] "RemoveContainer" containerID="e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d"
Jul 26 15:21:02.166685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.166671    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.166723 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:21:02.166710    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d\": container with ID starting with e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d not found: ID does not exist" containerID="e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d"
Jul 26 15:21:02.166758 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.166724    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d"} err="failed to get container status \"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d\": rpc error: code = NotFound desc = could not find container \"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d\": container with ID starting with e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d not found: ID does not exist"
Jul 26 15:21:02.166758 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.166736    2559 scope.go:117] "RemoveContainer" containerID="f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421"
Jul 26 15:21:02.166993 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:21:02.166971    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421\": container with ID starting with f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421 not found: ID does not exist" containerID="f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421"
Jul 26 15:21:02.167041 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.167001    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421"} err="failed to get container status \"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421\": rpc error: code = NotFound desc = could not find container \"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421\": container with ID starting with f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421 not found: ID does not exist"
Jul 26 15:21:02.167041 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.167022    2559 scope.go:117] "RemoveContainer" containerID="e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2"
Jul 26 15:21:02.167276 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:21:02.167253    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2\": container with ID starting with e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2 not found: ID does not exist" containerID="e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2"
Jul 26 15:21:02.167361 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.167284    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2"} err="failed to get container status \"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2\": rpc error: code = NotFound desc = could not find container \"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2\": container with ID starting with e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2 not found: ID does not exist"
Jul 26 15:21:02.167361 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.167306    2559 scope.go:117] "RemoveContainer" containerID="d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b"
Jul 26 15:21:02.167597 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:21:02.167576    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b\": container with ID starting with d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b not found: ID does not exist" containerID="d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b"
Jul 26 15:21:02.167656 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.167608    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b"} err="failed to get container status \"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b\": rpc error: code = NotFound desc = could not find container \"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b\": container with ID starting with d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b not found: ID does not exist"
Jul 26 15:21:02.167699 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.167661    2559 scope.go:117] "RemoveContainer" containerID="31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c"
Jul 26 15:21:02.167953 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:21:02.167936    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c\": container with ID starting with 31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c not found: ID does not exist" containerID="31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c"
Jul 26 15:21:02.167997 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.167959    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c"} err="failed to get container status \"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c\": rpc error: code = NotFound desc = could not find container \"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c\": container with ID starting with 31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c not found: ID does not exist"
Jul 26 15:21:02.167997 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.167973    2559 scope.go:117] "RemoveContainer" containerID="867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9"
Jul 26 15:21:02.168255 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.168234    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9"} err="failed to get container status \"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9\": rpc error: code = NotFound desc = could not find container \"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9\": container with ID starting with 867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9 not found: ID does not exist"
Jul 26 15:21:02.168309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.168257    2559 scope.go:117] "RemoveContainer" containerID="81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1"
Jul 26 15:21:02.168510 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.168492    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1"} err="failed to get container status \"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1\": rpc error: code = NotFound desc = could not find container \"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1\": container with ID starting with 81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1 not found: ID does not exist"
Jul 26 15:21:02.168560 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.168512    2559 scope.go:117] "RemoveContainer" containerID="e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d"
Jul 26 15:21:02.168762 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.168730    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d"} err="failed to get container status \"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d\": rpc error: code = NotFound desc = could not find container \"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d\": container with ID starting with e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d not found: ID does not exist"
Jul 26 15:21:02.168762 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.168763    2559 scope.go:117] "RemoveContainer" containerID="f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421"
Jul 26 15:21:02.169038 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.168835    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 26 15:21:02.169038 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.168848    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 26 15:21:02.169163 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.169098    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 26 15:21:02.169247 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.169087    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421"} err="failed to get container status \"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421\": rpc error: code = NotFound desc = could not find container \"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421\": container with ID starting with f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421 not found: ID does not exist"
Jul 26 15:21:02.169247 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.169197    2559 scope.go:117] "RemoveContainer" containerID="e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2"
Jul 26 15:21:02.169247 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.169245    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 26 15:21:02.169458 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.169372    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 26 15:21:02.169503 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.169465    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4fr2e752lc0r4\""
Jul 26 15:21:02.169580 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.169552    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 26 15:21:02.169580 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.169569    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 26 15:21:02.169688 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.169467    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pv29c\""
Jul 26 15:21:02.169741 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.169730    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 26 15:21:02.169925 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.169905    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 26 15:21:02.170156 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.170137    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 26 15:21:02.170266 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.170163    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 26 15:21:02.170330 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.170286    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 26 15:21:02.170575 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.170548    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2"} err="failed to get container status \"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2\": rpc error: code = NotFound desc = could not find container \"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2\": container with ID starting with e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2 not found: ID does not exist"
Jul 26 15:21:02.170575 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.170575    2559 scope.go:117] "RemoveContainer" containerID="d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b"
Jul 26 15:21:02.170939 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.170908    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b"} err="failed to get container status \"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b\": rpc error: code = NotFound desc = could not find container \"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b\": container with ID starting with d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b not found: ID does not exist"
Jul 26 15:21:02.170939 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.170928    2559 scope.go:117] "RemoveContainer" containerID="31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c"
Jul 26 15:21:02.171182 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.171158    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c"} err="failed to get container status \"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c\": rpc error: code = NotFound desc = could not find container \"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c\": container with ID starting with 31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c not found: ID does not exist"
Jul 26 15:21:02.171323 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.171185    2559 scope.go:117] "RemoveContainer" containerID="867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9"
Jul 26 15:21:02.171432 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.171412    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9"} err="failed to get container status \"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9\": rpc error: code = NotFound desc = could not find container \"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9\": container with ID starting with 867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9 not found: ID does not exist"
Jul 26 15:21:02.171553 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.171432    2559 scope.go:117] "RemoveContainer" containerID="81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1"
Jul 26 15:21:02.171696 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.171672    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1"} err="failed to get container status \"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1\": rpc error: code = NotFound desc = could not find container \"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1\": container with ID starting with 81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1 not found: ID does not exist"
Jul 26 15:21:02.171772 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.171699    2559 scope.go:117] "RemoveContainer" containerID="e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d"
Jul 26 15:21:02.172039 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.172013    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d"} err="failed to get container status \"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d\": rpc error: code = NotFound desc = could not find container \"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d\": container with ID starting with e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d not found: ID does not exist"
Jul 26 15:21:02.172121 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.172061    2559 scope.go:117] "RemoveContainer" containerID="f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421"
Jul 26 15:21:02.172361 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.172343    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421"} err="failed to get container status \"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421\": rpc error: code = NotFound desc = could not find container \"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421\": container with ID starting with f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421 not found: ID does not exist"
Jul 26 15:21:02.172435 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.172361    2559 scope.go:117] "RemoveContainer" containerID="e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2"
Jul 26 15:21:02.172639 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.172604    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2"} err="failed to get container status \"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2\": rpc error: code = NotFound desc = could not find container \"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2\": container with ID starting with e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2 not found: ID does not exist"
Jul 26 15:21:02.172717 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.172639    2559 scope.go:117] "RemoveContainer" containerID="d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b"
Jul 26 15:21:02.172845 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.172827    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 26 15:21:02.172905 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.172886    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b"} err="failed to get container status \"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b\": rpc error: code = NotFound desc = could not find container \"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b\": container with ID starting with d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b not found: ID does not exist"
Jul 26 15:21:02.172959 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.172909    2559 scope.go:117] "RemoveContainer" containerID="31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c"
Jul 26 15:21:02.173235 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.173178    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c"} err="failed to get container status \"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c\": rpc error: code = NotFound desc = could not find container \"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c\": container with ID starting with 31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c not found: ID does not exist"
Jul 26 15:21:02.173235 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.173230    2559 scope.go:117] "RemoveContainer" containerID="867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9"
Jul 26 15:21:02.173503 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.173482    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9"} err="failed to get container status \"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9\": rpc error: code = NotFound desc = could not find container \"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9\": container with ID starting with 867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9 not found: ID does not exist"
Jul 26 15:21:02.173503 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.173503    2559 scope.go:117] "RemoveContainer" containerID="81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1"
Jul 26 15:21:02.173889 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.173864    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1"} err="failed to get container status \"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1\": rpc error: code = NotFound desc = could not find container \"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1\": container with ID starting with 81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1 not found: ID does not exist"
Jul 26 15:21:02.174103 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.173891    2559 scope.go:117] "RemoveContainer" containerID="e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d"
Jul 26 15:21:02.174179 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.174154    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d"} err="failed to get container status \"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d\": rpc error: code = NotFound desc = could not find container \"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d\": container with ID starting with e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d not found: ID does not exist"
Jul 26 15:21:02.174270 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.174181    2559 scope.go:117] "RemoveContainer" containerID="f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421"
Jul 26 15:21:02.174464 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.174445    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421"} err="failed to get container status \"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421\": rpc error: code = NotFound desc = could not find container \"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421\": container with ID starting with f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421 not found: ID does not exist"
Jul 26 15:21:02.174464 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.174464    2559 scope.go:117] "RemoveContainer" containerID="e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2"
Jul 26 15:21:02.174731 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.174710    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2"} err="failed to get container status \"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2\": rpc error: code = NotFound desc = could not find container \"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2\": container with ID starting with e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2 not found: ID does not exist"
Jul 26 15:21:02.174731 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.174731    2559 scope.go:117] "RemoveContainer" containerID="d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b"
Jul 26 15:21:02.175082 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.174981    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b"} err="failed to get container status \"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b\": rpc error: code = NotFound desc = could not find container \"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b\": container with ID starting with d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b not found: ID does not exist"
Jul 26 15:21:02.175082 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.174999    2559 scope.go:117] "RemoveContainer" containerID="31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c"
Jul 26 15:21:02.175301 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.175272    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c"} err="failed to get container status \"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c\": rpc error: code = NotFound desc = could not find container \"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c\": container with ID starting with 31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c not found: ID does not exist"
Jul 26 15:21:02.175301 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.175301    2559 scope.go:117] "RemoveContainer" containerID="867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9"
Jul 26 15:21:02.175464 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.175342    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 26 15:21:02.175580 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.175562    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9"} err="failed to get container status \"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9\": rpc error: code = NotFound desc = could not find container \"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9\": container with ID starting with 867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9 not found: ID does not exist"
Jul 26 15:21:02.175621 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.175582    2559 scope.go:117] "RemoveContainer" containerID="81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1"
Jul 26 15:21:02.175808 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.175792    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1"} err="failed to get container status \"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1\": rpc error: code = NotFound desc = could not find container \"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1\": container with ID starting with 81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1 not found: ID does not exist"
Jul 26 15:21:02.175874 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.175809    2559 scope.go:117] "RemoveContainer" containerID="e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d"
Jul 26 15:21:02.176027 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.176010    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d"} err="failed to get container status \"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d\": rpc error: code = NotFound desc = could not find container \"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d\": container with ID starting with e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d not found: ID does not exist"
Jul 26 15:21:02.176027 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.176026    2559 scope.go:117] "RemoveContainer" containerID="f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421"
Jul 26 15:21:02.176264 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.176242    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421"} err="failed to get container status \"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421\": rpc error: code = NotFound desc = could not find container \"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421\": container with ID starting with f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421 not found: ID does not exist"
Jul 26 15:21:02.176264 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.176262    2559 scope.go:117] "RemoveContainer" containerID="e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2"
Jul 26 15:21:02.176535 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.176514    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2"} err="failed to get container status \"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2\": rpc error: code = NotFound desc = could not find container \"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2\": container with ID starting with e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2 not found: ID does not exist"
Jul 26 15:21:02.176535 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.176534    2559 scope.go:117] "RemoveContainer" containerID="d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b"
Jul 26 15:21:02.176760 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.176742    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b"} err="failed to get container status \"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b\": rpc error: code = NotFound desc = could not find container \"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b\": container with ID starting with d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b not found: ID does not exist"
Jul 26 15:21:02.176760 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.176758    2559 scope.go:117] "RemoveContainer" containerID="31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c"
Jul 26 15:21:02.176968 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.176950    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c"} err="failed to get container status \"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c\": rpc error: code = NotFound desc = could not find container \"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c\": container with ID starting with 31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c not found: ID does not exist"
Jul 26 15:21:02.176968 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.176967    2559 scope.go:117] "RemoveContainer" containerID="867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9"
Jul 26 15:21:02.177173 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.177157    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9"} err="failed to get container status \"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9\": rpc error: code = NotFound desc = could not find container \"867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9\": container with ID starting with 867240678d3f1db6d0171373bee70f2653576f6d101d215db84c482ef37a80f9 not found: ID does not exist"
Jul 26 15:21:02.177173 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.177172    2559 scope.go:117] "RemoveContainer" containerID="81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1"
Jul 26 15:21:02.177388 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.177369    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1"} err="failed to get container status \"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1\": rpc error: code = NotFound desc = could not find container \"81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1\": container with ID starting with 81b64495a70f2dedee09d69f6c9d78f0c3c0433e2e4716d705050861e7b4c7a1 not found: ID does not exist"
Jul 26 15:21:02.177434 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.177389    2559 scope.go:117] "RemoveContainer" containerID="e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d"
Jul 26 15:21:02.177647 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.177629    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d"} err="failed to get container status \"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d\": rpc error: code = NotFound desc = could not find container \"e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d\": container with ID starting with e3401b5b34a578ab1e508781658ce7c0f9d2313c8a7f48db7c12dfafeba0472d not found: ID does not exist"
Jul 26 15:21:02.177692 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.177648    2559 scope.go:117] "RemoveContainer" containerID="f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421"
Jul 26 15:21:02.177871 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.177854    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421"} err="failed to get container status \"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421\": rpc error: code = NotFound desc = could not find container \"f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421\": container with ID starting with f08b562b53495929f144cb5c59bbfc646f10bd7461732160bc6c96b591139421 not found: ID does not exist"
Jul 26 15:21:02.177910 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.177872    2559 scope.go:117] "RemoveContainer" containerID="e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2"
Jul 26 15:21:02.178071 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.178054    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2"} err="failed to get container status \"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2\": rpc error: code = NotFound desc = could not find container \"e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2\": container with ID starting with e2bbe2eb9251be5e1c18c2ad905b6b3ad32217c740aaf2335987e87331f2a6e2 not found: ID does not exist"
Jul 26 15:21:02.178153 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.178072    2559 scope.go:117] "RemoveContainer" containerID="d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b"
Jul 26 15:21:02.178322 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.178293    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b"} err="failed to get container status \"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b\": rpc error: code = NotFound desc = could not find container \"d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b\": container with ID starting with d34408e836129d0fd94eb56d470862a935427b2d0d06ce023665c74f8c00875b not found: ID does not exist"
Jul 26 15:21:02.178322 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.178315    2559 scope.go:117] "RemoveContainer" containerID="31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c"
Jul 26 15:21:02.178528 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.178514    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c"} err="failed to get container status \"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c\": rpc error: code = NotFound desc = could not find container \"31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c\": container with ID starting with 31429a4397d4394a119f6f1bb11febb0a46d52500e82868b9802d654480e773c not found: ID does not exist"
Jul 26 15:21:02.186817 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.182944    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 26 15:21:02.192353 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.192324    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc5bea94-f396-4d67-ae50-b070a870a64e" path="/var/lib/kubelet/pods/bc5bea94-f396-4d67-ae50-b070a870a64e/volumes"
Jul 26 15:21:02.270097 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270056    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270097 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270103    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270364 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270148    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270364 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270169    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270364 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270219    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270364 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270264    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4k5z\" (UniqueName: \"kubernetes.io/projected/40f799a7-ce63-4379-b2c0-6302be099f1f-kube-api-access-q4k5z\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270364 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270301    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270364 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270317    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270584 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270364    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270584 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270400    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270584 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270418    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/40f799a7-ce63-4379-b2c0-6302be099f1f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270584 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270441    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270584 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270457    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270584 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270474    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-config\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270584 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270496    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-web-config\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270584 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270538    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40f799a7-ce63-4379-b2c0-6302be099f1f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270584 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270566    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270584 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270583    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40f799a7-ce63-4379-b2c0-6302be099f1f-config-out\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270889 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270600    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.270889 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.270618    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.371444 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.371351    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-config\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.371444 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.371400    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-web-config\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.371444 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.371421    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40f799a7-ce63-4379-b2c0-6302be099f1f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.371444 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.371444    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.371725 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.371570    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40f799a7-ce63-4379-b2c0-6302be099f1f-config-out\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.371725 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.371621    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.371725 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.371651    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.371725 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.371682    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.371725 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.371711    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.371969 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.371750    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.371969 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.371773    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.371969 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.371821    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.371969 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.371849    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4k5z\" (UniqueName: \"kubernetes.io/projected/40f799a7-ce63-4379-b2c0-6302be099f1f-kube-api-access-q4k5z\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.371969 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.371878    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.371969 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.371906    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.371969 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.371941    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.372395 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.371974    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.372395 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.371997    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/40f799a7-ce63-4379-b2c0-6302be099f1f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.372395 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.372020    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.372395 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.372032    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.372395 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.372045    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.372395 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.372360    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/40f799a7-ce63-4379-b2c0-6302be099f1f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.372808 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.372785    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.373465 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.373441    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.374732 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.374322    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-web-config\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.374732 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.374369    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40f799a7-ce63-4379-b2c0-6302be099f1f-config-out\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.374732 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.374378    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.374732 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.374555    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-config\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.375016 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.374798    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.375149 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.375122    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.376412 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.376385    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.376520 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.376485    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.376773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.376725    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40f799a7-ce63-4379-b2c0-6302be099f1f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.377039 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.377009    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.377179 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.377161    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.377276 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.377172    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40f799a7-ce63-4379-b2c0-6302be099f1f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.377276 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.377254    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.377757 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.377732    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.377867 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.377844    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/40f799a7-ce63-4379-b2c0-6302be099f1f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.381820 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.381801    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4k5z\" (UniqueName: \"kubernetes.io/projected/40f799a7-ce63-4379-b2c0-6302be099f1f-kube-api-access-q4k5z\") pod \"prometheus-k8s-0\" (UID: \"40f799a7-ce63-4379-b2c0-6302be099f1f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.478926 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.478870    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:02.616804 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:02.616771    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 26 15:21:02.619741 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:21:02.619717    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40f799a7_ce63_4379_b2c0_6302be099f1f.slice/crio-5a3214ec78d1ca262393337d43d73a9841a281f54d197bade6d114afb3554545 WatchSource:0}: Error finding container 5a3214ec78d1ca262393337d43d73a9841a281f54d197bade6d114afb3554545: Status 404 returned error can't find the container with id 5a3214ec78d1ca262393337d43d73a9841a281f54d197bade6d114afb3554545
Jul 26 15:21:03.117301 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:03.117267    2559 generic.go:356] "Generic (PLEG): container finished" podID="40f799a7-ce63-4379-b2c0-6302be099f1f" containerID="d622f3ea29dc254c747f0e44f4978aa6d582317cdf46aa69e6f5227e604e37f5" exitCode=0
Jul 26 15:21:03.117724 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:03.117362    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40f799a7-ce63-4379-b2c0-6302be099f1f","Type":"ContainerDied","Data":"d622f3ea29dc254c747f0e44f4978aa6d582317cdf46aa69e6f5227e604e37f5"}
Jul 26 15:21:03.117724 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:03.117450    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40f799a7-ce63-4379-b2c0-6302be099f1f","Type":"ContainerStarted","Data":"5a3214ec78d1ca262393337d43d73a9841a281f54d197bade6d114afb3554545"}
Jul 26 15:21:04.127554 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:04.127513    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40f799a7-ce63-4379-b2c0-6302be099f1f","Type":"ContainerStarted","Data":"c9c31c09842260eb843dee14c253f8768ed20971fb28ff48631ecfd0f2506897"}
Jul 26 15:21:04.128112 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:04.128090    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40f799a7-ce63-4379-b2c0-6302be099f1f","Type":"ContainerStarted","Data":"e47ada416f82d822fcbfe69d1073bb8892388d12a8b25f868d7a0051edb95f44"}
Jul 26 15:21:04.128186 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:04.128138    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40f799a7-ce63-4379-b2c0-6302be099f1f","Type":"ContainerStarted","Data":"2bf628144d235e322397a38c84718ff384e7ddfcf3d7de0e8c09f753d5dd5487"}
Jul 26 15:21:04.128186 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:04.128168    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40f799a7-ce63-4379-b2c0-6302be099f1f","Type":"ContainerStarted","Data":"d97be70e7e71a56a577c59eb32e442c1329b356333f9044f4be812a650f71517"}
Jul 26 15:21:04.128299 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:04.128195    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40f799a7-ce63-4379-b2c0-6302be099f1f","Type":"ContainerStarted","Data":"a201e9be6a22eed25937dc8fafb8908374324fd7209d0d8e33a084ce37ebf445"}
Jul 26 15:21:04.128299 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:04.128245    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40f799a7-ce63-4379-b2c0-6302be099f1f","Type":"ContainerStarted","Data":"0fb33a19ad694815fb5d5bd93f3832af8883678065a232744ef2d85ff131cbae"}
Jul 26 15:21:04.155827 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:04.155776    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.137359557 podStartE2EDuration="2.155753653s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.394096ms" isStatefulPod=false podCreationTimestamp="2026-07-26 15:21:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:21:04.153498062 +0000 UTC m=+148.443330180" watchObservedRunningTime="2026-07-26 15:21:04.155753653 +0000 UTC m=+148.445585772"
Jul 26 15:21:05.427733 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:05.427694    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:21:07.479840 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:07.479805    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:21:16.812223 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:16.812165    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:21:16.868447 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:16.868416    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:21:16.926411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:16.926368    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:21:36.596480 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:36.596446    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:21:36.656807 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:21:36.656771    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:22:02.479226 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:22:02.479169    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:22:02.494389 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:22:02.494358    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:22:03.375624 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:22:03.375597    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:22:11.178330 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:22:11.178292    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:22:11.231162 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:22:11.231127    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:22:23.162565 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:22:23.162527    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:22:23.218178 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:22:23.218147    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:22:38.939647 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:22:38.939612    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:22:39.040965 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:22:39.040929    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:22:40.306253 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:22:40.306222    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:22:51.169658 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:22:51.169622    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:22:51.284186 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:22:51.284153    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:22:51.383808 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:22:51.383773    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:22:57.537710 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:22:57.537675    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:22:57.636144 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:22:57.636108    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:18.917538 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:18.917493    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:20.309821 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:20.309792    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:22.533660 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:22.533623    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:22.634391 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:22.634358    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:24.361811 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:24.361776    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:24.458116 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:24.458077    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:24.659761 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:24.659679    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:24.785726 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:24.785686    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:26.555106 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:26.555073    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:26.757325 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:26.757292    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:27.403245 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:27.403212    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wkksl"]
Jul 26 15:23:27.406492 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:27.406476    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkksl"
Jul 26 15:23:27.408273 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:27.408251    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 26 15:23:27.408457 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:27.408437    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 26 15:23:27.408706 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:27.408693    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-g75fl\""
Jul 26 15:23:27.413018 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:27.412994    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wkksl"]
Jul 26 15:23:27.540865 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:27.540831    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74ff2988-2cf1-4f8d-a5ee-8855c7738724-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wkksl\" (UID: \"74ff2988-2cf1-4f8d-a5ee-8855c7738724\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkksl"
Jul 26 15:23:27.541049 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:27.540890    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgrhl\" (UniqueName: \"kubernetes.io/projected/74ff2988-2cf1-4f8d-a5ee-8855c7738724-kube-api-access-lgrhl\") pod \"cert-manager-cainjector-559cdd8c87-wkksl\" (UID: \"74ff2988-2cf1-4f8d-a5ee-8855c7738724\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkksl"
Jul 26 15:23:27.641712 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:27.641678    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74ff2988-2cf1-4f8d-a5ee-8855c7738724-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wkksl\" (UID: \"74ff2988-2cf1-4f8d-a5ee-8855c7738724\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkksl"
Jul 26 15:23:27.642055 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:27.641726    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgrhl\" (UniqueName: \"kubernetes.io/projected/74ff2988-2cf1-4f8d-a5ee-8855c7738724-kube-api-access-lgrhl\") pod \"cert-manager-cainjector-559cdd8c87-wkksl\" (UID: \"74ff2988-2cf1-4f8d-a5ee-8855c7738724\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkksl"
Jul 26 15:23:27.649044 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:27.649019    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74ff2988-2cf1-4f8d-a5ee-8855c7738724-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wkksl\" (UID: \"74ff2988-2cf1-4f8d-a5ee-8855c7738724\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkksl"
Jul 26 15:23:27.649180 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:27.649144    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgrhl\" (UniqueName: \"kubernetes.io/projected/74ff2988-2cf1-4f8d-a5ee-8855c7738724-kube-api-access-lgrhl\") pod \"cert-manager-cainjector-559cdd8c87-wkksl\" (UID: \"74ff2988-2cf1-4f8d-a5ee-8855c7738724\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkksl"
Jul 26 15:23:27.734941 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:27.734851    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkksl"
Jul 26 15:23:27.859294 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:27.859269    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wkksl"]
Jul 26 15:23:27.862444 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:23:27.862411    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74ff2988_2cf1_4f8d_a5ee_8855c7738724.slice/crio-6e33a57e4e962ff8af37ffab50d942f8a65ae4d1e5fe4078f92e083dfc754de1 WatchSource:0}: Error finding container 6e33a57e4e962ff8af37ffab50d942f8a65ae4d1e5fe4078f92e083dfc754de1: Status 404 returned error can't find the container with id 6e33a57e4e962ff8af37ffab50d942f8a65ae4d1e5fe4078f92e083dfc754de1
Jul 26 15:23:28.700928 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:28.700868    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkksl" event={"ID":"74ff2988-2cf1-4f8d-a5ee-8855c7738724","Type":"ContainerStarted","Data":"6e33a57e4e962ff8af37ffab50d942f8a65ae4d1e5fe4078f92e083dfc754de1"}
Jul 26 15:23:29.534518 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:29.534455    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:29.631740 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:29.631680    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:31.711134 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:31.711057    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:31.719371 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:31.719336    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkksl" event={"ID":"74ff2988-2cf1-4f8d-a5ee-8855c7738724","Type":"ContainerStarted","Data":"01983c085c8fb6c82b992954d71646c8b05d1719a59fda9dccb84daf38bd1f48"}
Jul 26 15:23:31.737991 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:31.737933    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkksl" podStartSLOduration=1.675873927 podStartE2EDuration="4.737918007s" totalImagesPullingTime="3.06204408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:23:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:23:31.736771141 +0000 UTC m=+296.026603259" watchObservedRunningTime="2026-07-26 15:23:31.737918007 +0000 UTC m=+296.027750125"
Jul 26 15:23:31.811331 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:31.811294    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:34.572797 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:34.572762    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:34.673248 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:34.673196    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qpths"]
Jul 26 15:23:34.676644 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:34.676623    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-qpths"
Jul 26 15:23:34.678388 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:34.678369    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-kr7v4\""
Jul 26 15:23:34.685861 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:34.685839    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qpths"]
Jul 26 15:23:34.805813 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:34.805776    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhs6q\" (UniqueName: \"kubernetes.io/projected/d852dedc-2a89-4364-a9ce-b3cb99cafb03-kube-api-access-mhs6q\") pod \"cert-manager-55b4d954d5-qpths\" (UID: \"d852dedc-2a89-4364-a9ce-b3cb99cafb03\") " pod="cert-manager/cert-manager-55b4d954d5-qpths"
Jul 26 15:23:34.805980 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:34.805882    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d852dedc-2a89-4364-a9ce-b3cb99cafb03-bound-sa-token\") pod \"cert-manager-55b4d954d5-qpths\" (UID: \"d852dedc-2a89-4364-a9ce-b3cb99cafb03\") " pod="cert-manager/cert-manager-55b4d954d5-qpths"
Jul 26 15:23:34.907344 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:34.907238    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d852dedc-2a89-4364-a9ce-b3cb99cafb03-bound-sa-token\") pod \"cert-manager-55b4d954d5-qpths\" (UID: \"d852dedc-2a89-4364-a9ce-b3cb99cafb03\") " pod="cert-manager/cert-manager-55b4d954d5-qpths"
Jul 26 15:23:34.907344 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:34.907330    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhs6q\" (UniqueName: \"kubernetes.io/projected/d852dedc-2a89-4364-a9ce-b3cb99cafb03-kube-api-access-mhs6q\") pod \"cert-manager-55b4d954d5-qpths\" (UID: \"d852dedc-2a89-4364-a9ce-b3cb99cafb03\") " pod="cert-manager/cert-manager-55b4d954d5-qpths"
Jul 26 15:23:34.914846 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:34.914815    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d852dedc-2a89-4364-a9ce-b3cb99cafb03-bound-sa-token\") pod \"cert-manager-55b4d954d5-qpths\" (UID: \"d852dedc-2a89-4364-a9ce-b3cb99cafb03\") " pod="cert-manager/cert-manager-55b4d954d5-qpths"
Jul 26 15:23:34.915002 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:34.914983    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhs6q\" (UniqueName: \"kubernetes.io/projected/d852dedc-2a89-4364-a9ce-b3cb99cafb03-kube-api-access-mhs6q\") pod \"cert-manager-55b4d954d5-qpths\" (UID: \"d852dedc-2a89-4364-a9ce-b3cb99cafb03\") " pod="cert-manager/cert-manager-55b4d954d5-qpths"
Jul 26 15:23:34.985780 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:34.985740    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-qpths"
Jul 26 15:23:35.114356 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:35.114324    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qpths"]
Jul 26 15:23:35.117658 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:23:35.117633    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd852dedc_2a89_4364_a9ce_b3cb99cafb03.slice/crio-4f969ca01e705f7ee9cbc9a973b691bf4f5c85d19accff0478ae164b60e4e28c WatchSource:0}: Error finding container 4f969ca01e705f7ee9cbc9a973b691bf4f5c85d19accff0478ae164b60e4e28c: Status 404 returned error can't find the container with id 4f969ca01e705f7ee9cbc9a973b691bf4f5c85d19accff0478ae164b60e4e28c
Jul 26 15:23:35.736273 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:35.736239    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-qpths" event={"ID":"d852dedc-2a89-4364-a9ce-b3cb99cafb03","Type":"ContainerStarted","Data":"f361121fac885261861da58bfc201b61c08bc7cb92ebb23f57c790640f05a5c4"}
Jul 26 15:23:35.736273 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:35.736278    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-qpths" event={"ID":"d852dedc-2a89-4364-a9ce-b3cb99cafb03","Type":"ContainerStarted","Data":"4f969ca01e705f7ee9cbc9a973b691bf4f5c85d19accff0478ae164b60e4e28c"}
Jul 26 15:23:35.749860 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:35.749800    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-qpths" podStartSLOduration=1.749779351 podStartE2EDuration="1.749779351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:23:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:23:35.74961192 +0000 UTC m=+300.039444048" watchObservedRunningTime="2026-07-26 15:23:35.749779351 +0000 UTC m=+300.039611471"
Jul 26 15:23:35.940451 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:35.940412    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:36.039166 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:36.039086    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:36.115376 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:36.115347    2559 kubelet.go:1648] "Image garbage collection succeeded"
Jul 26 15:23:41.924572 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:41.924537    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:42.025426 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:42.025392    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:44.075282 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:44.075245    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:49.702570 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:49.702534    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:56.735439 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:56.735406    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:56.829174 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:56.829134    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:57.386326 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.386295    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:57.623286 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.623253    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:57.728464 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.728379    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:57.772103 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.772067    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"]
Jul 26 15:23:57.777171 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.777135    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:23:57.781381 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.781355    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 26 15:23:57.781535 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.781415    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 26 15:23:57.781535 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.781427    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-p2r5n\""
Jul 26 15:23:57.781535 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.781355    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 26 15:23:57.781535 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.781476    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 26 15:23:57.790520 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.790496    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"]
Jul 26 15:23:57.803830 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.803794    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b9601f1f-c7fc-41a8-9336-c44781094551-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79d786d858-cz6m4\" (UID: \"b9601f1f-c7fc-41a8-9336-c44781094551\") " pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:23:57.804001 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.803856    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9601f1f-c7fc-41a8-9336-c44781094551-tmp\") pod \"opendatahub-operator-controller-manager-79d786d858-cz6m4\" (UID: \"b9601f1f-c7fc-41a8-9336-c44781094551\") " pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:23:57.804001 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.803884    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9601f1f-c7fc-41a8-9336-c44781094551-webhook-cert\") pod \"opendatahub-operator-controller-manager-79d786d858-cz6m4\" (UID: \"b9601f1f-c7fc-41a8-9336-c44781094551\") " pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:23:57.804001 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.803904    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b9601f1f-c7fc-41a8-9336-c44781094551-manifests\") pod \"opendatahub-operator-controller-manager-79d786d858-cz6m4\" (UID: \"b9601f1f-c7fc-41a8-9336-c44781094551\") " pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:23:57.804001 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.803931    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q57lb\" (UniqueName: \"kubernetes.io/projected/b9601f1f-c7fc-41a8-9336-c44781094551-kube-api-access-q57lb\") pod \"opendatahub-operator-controller-manager-79d786d858-cz6m4\" (UID: \"b9601f1f-c7fc-41a8-9336-c44781094551\") " pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:23:57.885963 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.885933    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:57.905103 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.905067    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9601f1f-c7fc-41a8-9336-c44781094551-tmp\") pod \"opendatahub-operator-controller-manager-79d786d858-cz6m4\" (UID: \"b9601f1f-c7fc-41a8-9336-c44781094551\") " pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:23:57.905271 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.905115    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9601f1f-c7fc-41a8-9336-c44781094551-webhook-cert\") pod \"opendatahub-operator-controller-manager-79d786d858-cz6m4\" (UID: \"b9601f1f-c7fc-41a8-9336-c44781094551\") " pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:23:57.905271 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.905139    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b9601f1f-c7fc-41a8-9336-c44781094551-manifests\") pod \"opendatahub-operator-controller-manager-79d786d858-cz6m4\" (UID: \"b9601f1f-c7fc-41a8-9336-c44781094551\") " pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:23:57.905271 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.905162    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q57lb\" (UniqueName: \"kubernetes.io/projected/b9601f1f-c7fc-41a8-9336-c44781094551-kube-api-access-q57lb\") pod \"opendatahub-operator-controller-manager-79d786d858-cz6m4\" (UID: \"b9601f1f-c7fc-41a8-9336-c44781094551\") " pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:23:57.905271 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.905239    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b9601f1f-c7fc-41a8-9336-c44781094551-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79d786d858-cz6m4\" (UID: \"b9601f1f-c7fc-41a8-9336-c44781094551\") " pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:23:57.905631 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.905610    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b9601f1f-c7fc-41a8-9336-c44781094551-manifests\") pod \"opendatahub-operator-controller-manager-79d786d858-cz6m4\" (UID: \"b9601f1f-c7fc-41a8-9336-c44781094551\") " pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:23:57.907458 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.907430    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9601f1f-c7fc-41a8-9336-c44781094551-tmp\") pod \"opendatahub-operator-controller-manager-79d786d858-cz6m4\" (UID: \"b9601f1f-c7fc-41a8-9336-c44781094551\") " pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:23:57.907681 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.907663    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9601f1f-c7fc-41a8-9336-c44781094551-webhook-cert\") pod \"opendatahub-operator-controller-manager-79d786d858-cz6m4\" (UID: \"b9601f1f-c7fc-41a8-9336-c44781094551\") " pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:23:57.907762 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.907745    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b9601f1f-c7fc-41a8-9336-c44781094551-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79d786d858-cz6m4\" (UID: \"b9601f1f-c7fc-41a8-9336-c44781094551\") " pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:23:57.915765 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.915737    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q57lb\" (UniqueName: \"kubernetes.io/projected/b9601f1f-c7fc-41a8-9336-c44781094551-kube-api-access-q57lb\") pod \"opendatahub-operator-controller-manager-79d786d858-cz6m4\" (UID: \"b9601f1f-c7fc-41a8-9336-c44781094551\") " pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:23:57.986474 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:57.986372    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:58.089097 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:58.089059    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:23:58.090347 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:58.090315    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:58.195773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:58.195742    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:58.231134 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:58.231104    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"]
Jul 26 15:23:58.234546 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:23:58.234516    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9601f1f_c7fc_41a8_9336_c44781094551.slice/crio-36f36749aa27892b4b7105e6d75de656e8c6c1d06fd9f3502a68454557a99122 WatchSource:0}: Error finding container 36f36749aa27892b4b7105e6d75de656e8c6c1d06fd9f3502a68454557a99122: Status 404 returned error can't find the container with id 36f36749aa27892b4b7105e6d75de656e8c6c1d06fd9f3502a68454557a99122
Jul 26 15:23:58.236306 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:58.236291    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 15:23:58.290297 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:58.290262    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:58.392338 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:58.392298    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:58.491293 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:58.491176    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:23:58.829351 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:23:58.829320    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4" event={"ID":"b9601f1f-c7fc-41a8-9336-c44781094551","Type":"ContainerStarted","Data":"36f36749aa27892b4b7105e6d75de656e8c6c1d06fd9f3502a68454557a99122"}
Jul 26 15:24:01.847942 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:01.847907    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4" event={"ID":"b9601f1f-c7fc-41a8-9336-c44781094551","Type":"ContainerStarted","Data":"fd9e445b4031153b7b2d754655f236f5855d85ffccfebe6b1eaecb7199d80e38"}
Jul 26 15:24:02.853112 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:02.853080    2559 generic.go:356] "Generic (PLEG): container finished" podID="b9601f1f-c7fc-41a8-9336-c44781094551" containerID="fd9e445b4031153b7b2d754655f236f5855d85ffccfebe6b1eaecb7199d80e38" exitCode=0
Jul 26 15:24:02.853535 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:02.853164    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4" event={"ID":"b9601f1f-c7fc-41a8-9336-c44781094551","Type":"ContainerDied","Data":"fd9e445b4031153b7b2d754655f236f5855d85ffccfebe6b1eaecb7199d80e38"}
Jul 26 15:24:03.859994 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:03.859953    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4" event={"ID":"b9601f1f-c7fc-41a8-9336-c44781094551","Type":"ContainerStarted","Data":"4bcaf95ce51b43838ae13e7acf4f8be8499f9d917cfb2c21dad4c1823a586f85"}
Jul 26 15:24:03.860495 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:03.860068    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:24:03.882346 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:03.882296    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4" podStartSLOduration=2.477081046 podStartE2EDuration="6.882281572s" totalImagesPullingTime="3.402116315s" totalInitContainerRuntime="1.003084211s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:23:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:24:03.880657744 +0000 UTC m=+328.170489877" watchObservedRunningTime="2026-07-26 15:24:03.882281572 +0000 UTC m=+328.172113693"
Jul 26 15:24:08.380869 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:08.380822    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:08.488808 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:08.488777    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:08.894499 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:08.894389    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:08.992557 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:08.992507    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:14.866837 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:14.866803    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-79d786d858-cz6m4"
Jul 26 15:24:16.749754 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.749714    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"]
Jul 26 15:24:16.760131 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.760104    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"
Jul 26 15:24:16.762126 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.762101    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 26 15:24:16.762888 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.762866    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 26 15:24:16.763031 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.762978    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 26 15:24:16.763103 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.763072    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-72z57\""
Jul 26 15:24:16.763216 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.763188    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 26 15:24:16.763282 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.763237    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 26 15:24:16.769054 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.769024    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"]
Jul 26 15:24:16.873543 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.873506    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzdxd\" (UniqueName: \"kubernetes.io/projected/5ecd158c-4691-4c51-865a-09ecde033886-kube-api-access-kzdxd\") pod \"lws-controller-manager-7dd68d68f6-x7tvd\" (UID: \"5ecd158c-4691-4c51-865a-09ecde033886\") " pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"
Jul 26 15:24:16.873543 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.873543    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5ecd158c-4691-4c51-865a-09ecde033886-manager-config\") pod \"lws-controller-manager-7dd68d68f6-x7tvd\" (UID: \"5ecd158c-4691-4c51-865a-09ecde033886\") " pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"
Jul 26 15:24:16.873799 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.873570    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ecd158c-4691-4c51-865a-09ecde033886-cert\") pod \"lws-controller-manager-7dd68d68f6-x7tvd\" (UID: \"5ecd158c-4691-4c51-865a-09ecde033886\") " pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"
Jul 26 15:24:16.873799 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.873670    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5ecd158c-4691-4c51-865a-09ecde033886-metrics-cert\") pod \"lws-controller-manager-7dd68d68f6-x7tvd\" (UID: \"5ecd158c-4691-4c51-865a-09ecde033886\") " pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"
Jul 26 15:24:16.974351 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.974314    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ecd158c-4691-4c51-865a-09ecde033886-cert\") pod \"lws-controller-manager-7dd68d68f6-x7tvd\" (UID: \"5ecd158c-4691-4c51-865a-09ecde033886\") " pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"
Jul 26 15:24:16.974528 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.974369    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5ecd158c-4691-4c51-865a-09ecde033886-metrics-cert\") pod \"lws-controller-manager-7dd68d68f6-x7tvd\" (UID: \"5ecd158c-4691-4c51-865a-09ecde033886\") " pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"
Jul 26 15:24:16.974528 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.974419    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzdxd\" (UniqueName: \"kubernetes.io/projected/5ecd158c-4691-4c51-865a-09ecde033886-kube-api-access-kzdxd\") pod \"lws-controller-manager-7dd68d68f6-x7tvd\" (UID: \"5ecd158c-4691-4c51-865a-09ecde033886\") " pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"
Jul 26 15:24:16.974528 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.974439    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5ecd158c-4691-4c51-865a-09ecde033886-manager-config\") pod \"lws-controller-manager-7dd68d68f6-x7tvd\" (UID: \"5ecd158c-4691-4c51-865a-09ecde033886\") " pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"
Jul 26 15:24:16.975166 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.975142    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5ecd158c-4691-4c51-865a-09ecde033886-manager-config\") pod \"lws-controller-manager-7dd68d68f6-x7tvd\" (UID: \"5ecd158c-4691-4c51-865a-09ecde033886\") " pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"
Jul 26 15:24:16.976820 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.976796    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ecd158c-4691-4c51-865a-09ecde033886-cert\") pod \"lws-controller-manager-7dd68d68f6-x7tvd\" (UID: \"5ecd158c-4691-4c51-865a-09ecde033886\") " pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"
Jul 26 15:24:16.976911 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.976841    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5ecd158c-4691-4c51-865a-09ecde033886-metrics-cert\") pod \"lws-controller-manager-7dd68d68f6-x7tvd\" (UID: \"5ecd158c-4691-4c51-865a-09ecde033886\") " pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"
Jul 26 15:24:16.998548 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:16.998519    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzdxd\" (UniqueName: \"kubernetes.io/projected/5ecd158c-4691-4c51-865a-09ecde033886-kube-api-access-kzdxd\") pod \"lws-controller-manager-7dd68d68f6-x7tvd\" (UID: \"5ecd158c-4691-4c51-865a-09ecde033886\") " pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"
Jul 26 15:24:17.069904 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:17.069877    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"
Jul 26 15:24:17.084632 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:17.084600    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:17.245373 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:24:17.245342    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ecd158c_4691_4c51_865a_09ecde033886.slice/crio-47147d43c31018f1413cb3d16cae277bf45466a7ea5ecb82c369ad0cb57961f4 WatchSource:0}: Error finding container 47147d43c31018f1413cb3d16cae277bf45466a7ea5ecb82c369ad0cb57961f4: Status 404 returned error can't find the container with id 47147d43c31018f1413cb3d16cae277bf45466a7ea5ecb82c369ad0cb57961f4
Jul 26 15:24:17.254985 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:17.254959    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:17.256088 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:17.256067    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"]
Jul 26 15:24:17.859612 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:17.859580    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:17.926185 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:17.926147    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd" event={"ID":"5ecd158c-4691-4c51-865a-09ecde033886","Type":"ContainerStarted","Data":"47147d43c31018f1413cb3d16cae277bf45466a7ea5ecb82c369ad0cb57961f4"}
Jul 26 15:24:17.967041 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:17.966996    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:18.060041 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:18.059992    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:18.261312 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:18.261149    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:18.494874 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:18.494835    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:18.601518 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:18.601458    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:19.942233 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:19.942178    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd" event={"ID":"5ecd158c-4691-4c51-865a-09ecde033886","Type":"ContainerStarted","Data":"8bba0c83644356f52ab750bcbbf0a661eb29524749d2338765c8647abe0407aa"}
Jul 26 15:24:19.942775 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:19.942326    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"
Jul 26 15:24:19.960050 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:19.959985    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd" podStartSLOduration=1.484946767 podStartE2EDuration="3.959963469s" totalImagesPullingTime="2.475016702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:24:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:24:19.95746773 +0000 UTC m=+344.247299845" watchObservedRunningTime="2026-07-26 15:24:19.959963469 +0000 UTC m=+344.249795590"
Jul 26 15:24:20.592870 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.592835    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:20.645274 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.645245    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6dc54b99bb-9tw42"]
Jul 26 15:24:20.648789 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.648768    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6dc54b99bb-9tw42"
Jul 26 15:24:20.650801 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.650774    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 26 15:24:20.650956 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.650805    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 26 15:24:20.651071 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.651058    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 26 15:24:20.651626 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.651386    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 26 15:24:20.651626 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.651575    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-fmsct\""
Jul 26 15:24:20.659115 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.659095    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6dc54b99bb-9tw42"]
Jul 26 15:24:20.709331 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.709292    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1afad906-4350-4409-a472-c8b30df96f7c-tmp\") pod \"kube-auth-proxy-6dc54b99bb-9tw42\" (UID: \"1afad906-4350-4409-a472-c8b30df96f7c\") " pod="openshift-ingress/kube-auth-proxy-6dc54b99bb-9tw42"
Jul 26 15:24:20.709331 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.709333    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1afad906-4350-4409-a472-c8b30df96f7c-tls-certs\") pod \"kube-auth-proxy-6dc54b99bb-9tw42\" (UID: \"1afad906-4350-4409-a472-c8b30df96f7c\") " pod="openshift-ingress/kube-auth-proxy-6dc54b99bb-9tw42"
Jul 26 15:24:20.709530 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.709383    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mks25\" (UniqueName: \"kubernetes.io/projected/1afad906-4350-4409-a472-c8b30df96f7c-kube-api-access-mks25\") pod \"kube-auth-proxy-6dc54b99bb-9tw42\" (UID: \"1afad906-4350-4409-a472-c8b30df96f7c\") " pod="openshift-ingress/kube-auth-proxy-6dc54b99bb-9tw42"
Jul 26 15:24:20.812270 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.811716    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1afad906-4350-4409-a472-c8b30df96f7c-tmp\") pod \"kube-auth-proxy-6dc54b99bb-9tw42\" (UID: \"1afad906-4350-4409-a472-c8b30df96f7c\") " pod="openshift-ingress/kube-auth-proxy-6dc54b99bb-9tw42"
Jul 26 15:24:20.812270 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.811766    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1afad906-4350-4409-a472-c8b30df96f7c-tls-certs\") pod \"kube-auth-proxy-6dc54b99bb-9tw42\" (UID: \"1afad906-4350-4409-a472-c8b30df96f7c\") " pod="openshift-ingress/kube-auth-proxy-6dc54b99bb-9tw42"
Jul 26 15:24:20.812270 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.811798    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mks25\" (UniqueName: \"kubernetes.io/projected/1afad906-4350-4409-a472-c8b30df96f7c-kube-api-access-mks25\") pod \"kube-auth-proxy-6dc54b99bb-9tw42\" (UID: \"1afad906-4350-4409-a472-c8b30df96f7c\") " pod="openshift-ingress/kube-auth-proxy-6dc54b99bb-9tw42"
Jul 26 15:24:20.814387 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.814358    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1afad906-4350-4409-a472-c8b30df96f7c-tmp\") pod \"kube-auth-proxy-6dc54b99bb-9tw42\" (UID: \"1afad906-4350-4409-a472-c8b30df96f7c\") " pod="openshift-ingress/kube-auth-proxy-6dc54b99bb-9tw42"
Jul 26 15:24:20.814534 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.814501    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1afad906-4350-4409-a472-c8b30df96f7c-tls-certs\") pod \"kube-auth-proxy-6dc54b99bb-9tw42\" (UID: \"1afad906-4350-4409-a472-c8b30df96f7c\") " pod="openshift-ingress/kube-auth-proxy-6dc54b99bb-9tw42"
Jul 26 15:24:20.815089 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.815069    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:20.818627 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.818607    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mks25\" (UniqueName: \"kubernetes.io/projected/1afad906-4350-4409-a472-c8b30df96f7c-kube-api-access-mks25\") pod \"kube-auth-proxy-6dc54b99bb-9tw42\" (UID: \"1afad906-4350-4409-a472-c8b30df96f7c\") " pod="openshift-ingress/kube-auth-proxy-6dc54b99bb-9tw42"
Jul 26 15:24:20.961343 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:20.961255    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6dc54b99bb-9tw42"
Jul 26 15:24:21.109939 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:21.109909    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6dc54b99bb-9tw42"]
Jul 26 15:24:21.111108 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:24:21.111082    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1afad906_4350_4409_a472_c8b30df96f7c.slice/crio-3c78dd62722c75a9275f7cba50133a57d4222304b84ef92e4e9e75982f06c676 WatchSource:0}: Error finding container 3c78dd62722c75a9275f7cba50133a57d4222304b84ef92e4e9e75982f06c676: Status 404 returned error can't find the container with id 3c78dd62722c75a9275f7cba50133a57d4222304b84ef92e4e9e75982f06c676
Jul 26 15:24:21.722446 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:21.722382    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:21.779766 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:21.779731    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:21.955513 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:21.955472    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6dc54b99bb-9tw42" event={"ID":"1afad906-4350-4409-a472-c8b30df96f7c","Type":"ContainerStarted","Data":"3c78dd62722c75a9275f7cba50133a57d4222304b84ef92e4e9e75982f06c676"}
Jul 26 15:24:22.624600 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:22.624569    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:23.967125 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:23.967089    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6dc54b99bb-9tw42" event={"ID":"1afad906-4350-4409-a472-c8b30df96f7c","Type":"ContainerStarted","Data":"75d8ce4d181467c87e4d81713b60d7df3ee5825e6db55f0863b4aa7781dd3594"}
Jul 26 15:24:23.982143 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:23.982094    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6dc54b99bb-9tw42" podStartSLOduration=1.590596175 podStartE2EDuration="3.982077188s" totalImagesPullingTime="2.391481013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:24:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:24:23.981275557 +0000 UTC m=+348.271107677" watchObservedRunningTime="2026-07-26 15:24:23.982077188 +0000 UTC m=+348.271909306"
Jul 26 15:24:25.661049 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:25.661018    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:25.754455 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:25.754423    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:26.549820 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:26.549772    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:26.774616 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:26.774583    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:27.218489 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:27.218444    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:28.311157 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:28.311066    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:28.422499 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:28.422467    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:28.605927 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:28.605834    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:28.782515 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:28.782484    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:28.873411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:28.873310    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:24:30.950114 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:30.950084    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7dd68d68f6-x7tvd"
Jul 26 15:24:59.910026 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:24:59.909991    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:25:00.010474 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:25:00.010439    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:25:18.589584 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:25:18.589544    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:25:18.690857 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:25:18.690826    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:25:18.787351 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:25:18.787321    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:25:19.517643 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:25:19.517599    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:25:19.718267 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:25:19.718238    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:25:41.799498 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:25:41.799460    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:25:45.379303 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:25:45.379256    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:25:48.180268 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:25:48.180233    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:25:48.777084 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:25:48.777049    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:25:51.038710 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:25:51.038676    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:25:52.177519 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:25:52.177484    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:25:55.805350 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:25:55.805312    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:25:57.121519 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:25:57.121482    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:25:58.593913 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:25:58.593832    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:25:58.699548 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:25:58.699522    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:25:59.004820 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:25:59.004743    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:01.625464 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:01.625421    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:01.763925 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:01.763896    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:05.736413 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.736377    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zdpp9"]
Jul 26 15:26:05.739419 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.739396    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zdpp9"
Jul 26 15:26:05.746711 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.746690    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-d546b\""
Jul 26 15:26:05.746837 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.746746    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 26 15:26:05.747400 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.747384    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 26 15:26:05.747400 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.747391    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 26 15:26:05.747540 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.747460    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 26 15:26:05.772825 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.772786    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zdpp9"]
Jul 26 15:26:05.778556 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.778523    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:05.789863 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.789834    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zdpp9\" (UID: \"4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zdpp9"
Jul 26 15:26:05.790023 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.789867    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hdtv\" (UniqueName: \"kubernetes.io/projected/4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef-kube-api-access-9hdtv\") pod \"kuadrant-console-plugin-66b5774985-zdpp9\" (UID: \"4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zdpp9"
Jul 26 15:26:05.790023 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.789957    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zdpp9\" (UID: \"4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zdpp9"
Jul 26 15:26:05.870508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.870477    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:05.890952 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.890916    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zdpp9\" (UID: \"4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zdpp9"
Jul 26 15:26:05.890952 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.890953    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hdtv\" (UniqueName: \"kubernetes.io/projected/4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef-kube-api-access-9hdtv\") pod \"kuadrant-console-plugin-66b5774985-zdpp9\" (UID: \"4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zdpp9"
Jul 26 15:26:05.891162 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.891029    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zdpp9\" (UID: \"4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zdpp9"
Jul 26 15:26:05.891736 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.891715    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zdpp9\" (UID: \"4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zdpp9"
Jul 26 15:26:05.893419 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.893398    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zdpp9\" (UID: \"4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zdpp9"
Jul 26 15:26:05.898291 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:05.898265    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hdtv\" (UniqueName: \"kubernetes.io/projected/4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef-kube-api-access-9hdtv\") pod \"kuadrant-console-plugin-66b5774985-zdpp9\" (UID: \"4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zdpp9"
Jul 26 15:26:06.049133 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:06.049040    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zdpp9"
Jul 26 15:26:06.186386 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:06.186333    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zdpp9"]
Jul 26 15:26:06.189368 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:26:06.189341    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c1d1bc3_44e5_4b22_9d10_726d0af0c9ef.slice/crio-1c000214fd93129bf512e9557f6b2c796000b6c216b2548920888d13a38b011a WatchSource:0}: Error finding container 1c000214fd93129bf512e9557f6b2c796000b6c216b2548920888d13a38b011a: Status 404 returned error can't find the container with id 1c000214fd93129bf512e9557f6b2c796000b6c216b2548920888d13a38b011a
Jul 26 15:26:06.424163 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:06.424124    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zdpp9" event={"ID":"4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef","Type":"ContainerStarted","Data":"1c000214fd93129bf512e9557f6b2c796000b6c216b2548920888d13a38b011a"}
Jul 26 15:26:06.461034 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:06.461002    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:06.562190 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:06.562156    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:06.988507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:06.988435    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:07.092566 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:07.092523    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:07.385352 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:07.385320    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:07.441978 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:07.441935    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:10.975753 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:10.975713    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:11.028439 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:11.028402    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:13.162955 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:13.162918    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:15.147905 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.147868    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"]
Jul 26 15:26:15.155038 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.155008    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"
Jul 26 15:26:15.157133 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.157106    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hwfpn\""
Jul 26 15:26:15.165081 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.164603    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"]
Jul 26 15:26:15.280688 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.280645    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blbrw\" (UniqueName: \"kubernetes.io/projected/0cea1b4d-8b3f-4a51-b71e-85f4d66988a4-kube-api-access-blbrw\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-24tpg\" (UID: \"0cea1b4d-8b3f-4a51-b71e-85f4d66988a4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"
Jul 26 15:26:15.280872 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.280740    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0cea1b4d-8b3f-4a51-b71e-85f4d66988a4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-24tpg\" (UID: \"0cea1b4d-8b3f-4a51-b71e-85f4d66988a4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"
Jul 26 15:26:15.381715 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.381679    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blbrw\" (UniqueName: \"kubernetes.io/projected/0cea1b4d-8b3f-4a51-b71e-85f4d66988a4-kube-api-access-blbrw\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-24tpg\" (UID: \"0cea1b4d-8b3f-4a51-b71e-85f4d66988a4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"
Jul 26 15:26:15.381920 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.381756    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0cea1b4d-8b3f-4a51-b71e-85f4d66988a4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-24tpg\" (UID: \"0cea1b4d-8b3f-4a51-b71e-85f4d66988a4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"
Jul 26 15:26:15.382129 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.382108    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0cea1b4d-8b3f-4a51-b71e-85f4d66988a4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-24tpg\" (UID: \"0cea1b4d-8b3f-4a51-b71e-85f4d66988a4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"
Jul 26 15:26:15.390161 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.390135    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blbrw\" (UniqueName: \"kubernetes.io/projected/0cea1b4d-8b3f-4a51-b71e-85f4d66988a4-kube-api-access-blbrw\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-24tpg\" (UID: \"0cea1b4d-8b3f-4a51-b71e-85f4d66988a4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"
Jul 26 15:26:15.469829 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.469751    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"
Jul 26 15:26:15.777813 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.777727    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"]
Jul 26 15:26:15.783949 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.783915    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"]
Jul 26 15:26:15.799074 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.799038    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7"]
Jul 26 15:26:15.804518 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.804496    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7"
Jul 26 15:26:15.815098 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.814506    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7"]
Jul 26 15:26:15.886804 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.886772    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vchmh\" (UniqueName: \"kubernetes.io/projected/f73033c8-003d-45f4-aafe-661bd982693a-kube-api-access-vchmh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wznj7\" (UID: \"f73033c8-003d-45f4-aafe-661bd982693a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7"
Jul 26 15:26:15.886976 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.886831    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f73033c8-003d-45f4-aafe-661bd982693a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wznj7\" (UID: \"f73033c8-003d-45f4-aafe-661bd982693a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7"
Jul 26 15:26:15.987630 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.987594    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vchmh\" (UniqueName: \"kubernetes.io/projected/f73033c8-003d-45f4-aafe-661bd982693a-kube-api-access-vchmh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wznj7\" (UID: \"f73033c8-003d-45f4-aafe-661bd982693a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7"
Jul 26 15:26:15.987812 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.987665    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f73033c8-003d-45f4-aafe-661bd982693a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wznj7\" (UID: \"f73033c8-003d-45f4-aafe-661bd982693a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7"
Jul 26 15:26:15.988126 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.988107    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f73033c8-003d-45f4-aafe-661bd982693a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wznj7\" (UID: \"f73033c8-003d-45f4-aafe-661bd982693a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7"
Jul 26 15:26:15.995999 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:15.995968    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vchmh\" (UniqueName: \"kubernetes.io/projected/f73033c8-003d-45f4-aafe-661bd982693a-kube-api-access-vchmh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wznj7\" (UID: \"f73033c8-003d-45f4-aafe-661bd982693a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7"
Jul 26 15:26:16.119922 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:16.119878    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7"
Jul 26 15:26:17.094583 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:17.094549    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:17.196667 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:17.196612    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:17.300381 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:17.300343    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:19.171524 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:19.171488    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:19.272969 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:19.272934    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:22.575193 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:22.575152    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:22.676086 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:22.676043    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:27.366857 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:27.366782    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:27.418434 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:27.418402    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:27.968049 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:27.968013    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:28.069541 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:28.069511    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:29.596173 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:29.596150    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7"]
Jul 26 15:26:29.598423 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:26:29.598387    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf73033c8_003d_45f4_aafe_661bd982693a.slice/crio-ccdce5f56c3f917469632d49ce8c632323f6ec4bde230087dced8c8bb4a4b9e1 WatchSource:0}: Error finding container ccdce5f56c3f917469632d49ce8c632323f6ec4bde230087dced8c8bb4a4b9e1: Status 404 returned error can't find the container with id ccdce5f56c3f917469632d49ce8c632323f6ec4bde230087dced8c8bb4a4b9e1
Jul 26 15:26:30.551333 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:30.551249    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7" event={"ID":"f73033c8-003d-45f4-aafe-661bd982693a","Type":"ContainerStarted","Data":"ccdce5f56c3f917469632d49ce8c632323f6ec4bde230087dced8c8bb4a4b9e1"}
Jul 26 15:26:30.552765 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:30.552741    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zdpp9" event={"ID":"4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef","Type":"ContainerStarted","Data":"ee2d7a230b88027b544fb6c1605927fa260f7723d3c3b9ea8996cbb282dac17f"}
Jul 26 15:26:30.569537 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:30.569468    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zdpp9" podStartSLOduration=1.913095902 podStartE2EDuration="25.569449377s" totalImagesPullingTime="23.656353475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:26:30.568074572 +0000 UTC m=+474.857906690" watchObservedRunningTime="2026-07-26 15:26:30.569449377 +0000 UTC m=+474.859281499"
Jul 26 15:26:31.265473 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:31.265436    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:31.331688 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:31.331654    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:32.002957 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:26:32.002908    2559 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-24tpg_kuadrant-system_0cea1b4d-8b3f-4a51-b71e-85f4d66988a4_0(34293cb19fe8d9d1d9a9f01c888b5e854e7659f77db3a8b88e3b54ba1c14d0c4): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-24tpg to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 26 15:26:32.003073 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:26:32.002989    2559 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-24tpg_kuadrant-system_0cea1b4d-8b3f-4a51-b71e-85f4d66988a4_0(34293cb19fe8d9d1d9a9f01c888b5e854e7659f77db3a8b88e3b54ba1c14d0c4): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-24tpg to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"
Jul 26 15:26:32.167171 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:32.167138    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:32.220657 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:32.220615    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:32.562921 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:32.562890    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"
Jul 26 15:26:32.564719 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:26:32.564689    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-24tpg\" is forbidden: User \"system:node:ip-10-0-139-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-110.ec2.internal' and this object" podUID="0cea1b4d-8b3f-4a51-b71e-85f4d66988a4" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"
Jul 26 15:26:32.602115 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:32.602082    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"
Jul 26 15:26:32.603927 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:26:32.603892    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-24tpg\" is forbidden: User \"system:node:ip-10-0-139-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-110.ec2.internal' and this object" podUID="0cea1b4d-8b3f-4a51-b71e-85f4d66988a4" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"
Jul 26 15:26:32.748933 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:32.748896    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blbrw\" (UniqueName: \"kubernetes.io/projected/0cea1b4d-8b3f-4a51-b71e-85f4d66988a4-kube-api-access-blbrw\") pod \"0cea1b4d-8b3f-4a51-b71e-85f4d66988a4\" (UID: \"0cea1b4d-8b3f-4a51-b71e-85f4d66988a4\") "
Jul 26 15:26:32.749116 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:32.748951    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0cea1b4d-8b3f-4a51-b71e-85f4d66988a4-extensions-socket-volume\") pod \"0cea1b4d-8b3f-4a51-b71e-85f4d66988a4\" (UID: \"0cea1b4d-8b3f-4a51-b71e-85f4d66988a4\") "
Jul 26 15:26:32.749328 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:32.749302    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cea1b4d-8b3f-4a51-b71e-85f4d66988a4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0cea1b4d-8b3f-4a51-b71e-85f4d66988a4" (UID: "0cea1b4d-8b3f-4a51-b71e-85f4d66988a4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 15:26:32.751041 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:32.751012    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cea1b4d-8b3f-4a51-b71e-85f4d66988a4-kube-api-access-blbrw" (OuterVolumeSpecName: "kube-api-access-blbrw") pod "0cea1b4d-8b3f-4a51-b71e-85f4d66988a4" (UID: "0cea1b4d-8b3f-4a51-b71e-85f4d66988a4"). InnerVolumeSpecName "kube-api-access-blbrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:26:32.849710 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:32.849625    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blbrw\" (UniqueName: \"kubernetes.io/projected/0cea1b4d-8b3f-4a51-b71e-85f4d66988a4-kube-api-access-blbrw\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:26:32.849710 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:32.849658    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0cea1b4d-8b3f-4a51-b71e-85f4d66988a4-extensions-socket-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:26:33.568493 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:33.568451    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"
Jul 26 15:26:33.570566 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:26:33.570522    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-24tpg\" is forbidden: User \"system:node:ip-10-0-139-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-110.ec2.internal' and this object" podUID="0cea1b4d-8b3f-4a51-b71e-85f4d66988a4" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"
Jul 26 15:26:33.577981 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:26:33.577946    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-24tpg\" is forbidden: User \"system:node:ip-10-0-139-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-110.ec2.internal' and this object" podUID="0cea1b4d-8b3f-4a51-b71e-85f4d66988a4" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-24tpg"
Jul 26 15:26:34.194324 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:34.194290    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cea1b4d-8b3f-4a51-b71e-85f4d66988a4" path="/var/lib/kubelet/pods/0cea1b4d-8b3f-4a51-b71e-85f4d66988a4/volumes"
Jul 26 15:26:35.581623 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:35.581573    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7" event={"ID":"f73033c8-003d-45f4-aafe-661bd982693a","Type":"ContainerStarted","Data":"49de5177bb6ce851eb229ac2652935a4486fcd47b98bcd65c374b6d5db0cb1db"}
Jul 26 15:26:35.582072 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:35.581685    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7"
Jul 26 15:26:35.598130 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:35.598072    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7" podStartSLOduration=15.124509584 podStartE2EDuration="20.598055309s" totalImagesPullingTime="5.473545725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:26:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:26:35.596454503 +0000 UTC m=+479.886286623" watchObservedRunningTime="2026-07-26 15:26:35.598055309 +0000 UTC m=+479.887887428"
Jul 26 15:26:39.181498 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:39.181460    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:39.286422 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:39.286381    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:42.392157 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:42.392117    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:42.494475 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:42.494446    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:46.589006 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:46.588967    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7"
Jul 26 15:26:47.102198 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:47.102113    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:47.204094 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:47.204051    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:47.303756 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:47.303721    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:51.996541 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:51.996495    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:52.053851 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:52.053817    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:26:59.381823 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:59.381786    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7"]
Jul 26 15:26:59.382354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:59.382091    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7" podUID="f73033c8-003d-45f4-aafe-661bd982693a" containerName="manager" containerID="cri-o://49de5177bb6ce851eb229ac2652935a4486fcd47b98bcd65c374b6d5db0cb1db" gracePeriod=10
Jul 26 15:26:59.691758 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:59.691720    2559 generic.go:356] "Generic (PLEG): container finished" podID="f73033c8-003d-45f4-aafe-661bd982693a" containerID="49de5177bb6ce851eb229ac2652935a4486fcd47b98bcd65c374b6d5db0cb1db" exitCode=0
Jul 26 15:26:59.691758 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:59.691755    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7" event={"ID":"f73033c8-003d-45f4-aafe-661bd982693a","Type":"ContainerDied","Data":"49de5177bb6ce851eb229ac2652935a4486fcd47b98bcd65c374b6d5db0cb1db"}
Jul 26 15:26:59.692005 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:59.691795    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7" event={"ID":"f73033c8-003d-45f4-aafe-661bd982693a","Type":"ContainerDied","Data":"ccdce5f56c3f917469632d49ce8c632323f6ec4bde230087dced8c8bb4a4b9e1"}
Jul 26 15:26:59.692005 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:59.691812    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ccdce5f56c3f917469632d49ce8c632323f6ec4bde230087dced8c8bb4a4b9e1"
Jul 26 15:26:59.700755 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:59.700721    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7"
Jul 26 15:26:59.778366 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:59.778310    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f73033c8-003d-45f4-aafe-661bd982693a-extensions-socket-volume\") pod \"f73033c8-003d-45f4-aafe-661bd982693a\" (UID: \"f73033c8-003d-45f4-aafe-661bd982693a\") "
Jul 26 15:26:59.778558 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:59.778387    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vchmh\" (UniqueName: \"kubernetes.io/projected/f73033c8-003d-45f4-aafe-661bd982693a-kube-api-access-vchmh\") pod \"f73033c8-003d-45f4-aafe-661bd982693a\" (UID: \"f73033c8-003d-45f4-aafe-661bd982693a\") "
Jul 26 15:26:59.778758 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:59.778733    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f73033c8-003d-45f4-aafe-661bd982693a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f73033c8-003d-45f4-aafe-661bd982693a" (UID: "f73033c8-003d-45f4-aafe-661bd982693a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 15:26:59.780611 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:59.780571    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f73033c8-003d-45f4-aafe-661bd982693a-kube-api-access-vchmh" (OuterVolumeSpecName: "kube-api-access-vchmh") pod "f73033c8-003d-45f4-aafe-661bd982693a" (UID: "f73033c8-003d-45f4-aafe-661bd982693a"). InnerVolumeSpecName "kube-api-access-vchmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:26:59.879726 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:59.879686    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vchmh\" (UniqueName: \"kubernetes.io/projected/f73033c8-003d-45f4-aafe-661bd982693a-kube-api-access-vchmh\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:26:59.879726 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:26:59.879719    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f73033c8-003d-45f4-aafe-661bd982693a-extensions-socket-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:27:00.204723 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:00.204693    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:00.297564 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:00.297518    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:00.696045 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:00.696013    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7"
Jul 26 15:27:00.710894 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:00.710854    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7"]
Jul 26 15:27:00.714741 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:00.714709    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wznj7"]
Jul 26 15:27:02.193474 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:02.193445    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f73033c8-003d-45f4-aafe-661bd982693a" path="/var/lib/kubelet/pods/f73033c8-003d-45f4-aafe-661bd982693a/volumes"
Jul 26 15:27:03.016914 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:03.016885    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:03.115309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:03.115269    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:05.504847 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.504811    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:05.513400 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.513369    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"]
Jul 26 15:27:05.513717 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.513705    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f73033c8-003d-45f4-aafe-661bd982693a" containerName="manager"
Jul 26 15:27:05.513771 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.513718    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f73033c8-003d-45f4-aafe-661bd982693a" containerName="manager"
Jul 26 15:27:05.513811 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.513782    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f73033c8-003d-45f4-aafe-661bd982693a" containerName="manager"
Jul 26 15:27:05.533329 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.533299    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"]
Jul 26 15:27:05.533491 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.533417    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"
Jul 26 15:27:05.535416 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.535392    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 26 15:27:05.610297 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.610262    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"]
Jul 26 15:27:05.616470 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.616441    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:05.637934 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.637899    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94twh\" (UniqueName: \"kubernetes.io/projected/081c1bc3-9b98-4342-b21d-d76c5d208bd5-kube-api-access-94twh\") pod \"limitador-limitador-64c94fd64f-lwfsl\" (UID: \"081c1bc3-9b98-4342-b21d-d76c5d208bd5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"
Jul 26 15:27:05.638091 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.637969    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/081c1bc3-9b98-4342-b21d-d76c5d208bd5-config-file\") pod \"limitador-limitador-64c94fd64f-lwfsl\" (UID: \"081c1bc3-9b98-4342-b21d-d76c5d208bd5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"
Jul 26 15:27:05.669758 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.669728    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:05.739111 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.739070    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/081c1bc3-9b98-4342-b21d-d76c5d208bd5-config-file\") pod \"limitador-limitador-64c94fd64f-lwfsl\" (UID: \"081c1bc3-9b98-4342-b21d-d76c5d208bd5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"
Jul 26 15:27:05.739324 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.739129    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94twh\" (UniqueName: \"kubernetes.io/projected/081c1bc3-9b98-4342-b21d-d76c5d208bd5-kube-api-access-94twh\") pod \"limitador-limitador-64c94fd64f-lwfsl\" (UID: \"081c1bc3-9b98-4342-b21d-d76c5d208bd5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"
Jul 26 15:27:05.739779 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.739760    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/081c1bc3-9b98-4342-b21d-d76c5d208bd5-config-file\") pod \"limitador-limitador-64c94fd64f-lwfsl\" (UID: \"081c1bc3-9b98-4342-b21d-d76c5d208bd5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"
Jul 26 15:27:05.746511 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.746485    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94twh\" (UniqueName: \"kubernetes.io/projected/081c1bc3-9b98-4342-b21d-d76c5d208bd5-kube-api-access-94twh\") pod \"limitador-limitador-64c94fd64f-lwfsl\" (UID: \"081c1bc3-9b98-4342-b21d-d76c5d208bd5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"
Jul 26 15:27:05.843555 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.843521    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"
Jul 26 15:27:05.979846 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:05.979810    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"]
Jul 26 15:27:05.982805 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:27:05.982758    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081c1bc3_9b98_4342_b21d_d76c5d208bd5.slice/crio-f65ba0044728ffca02d61e2cd9a6fa7a29a8cd4be6eafa46a0f98cd053457e3c WatchSource:0}: Error finding container f65ba0044728ffca02d61e2cd9a6fa7a29a8cd4be6eafa46a0f98cd053457e3c: Status 404 returned error can't find the container with id f65ba0044728ffca02d61e2cd9a6fa7a29a8cd4be6eafa46a0f98cd053457e3c
Jul 26 15:27:06.354659 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:06.354626    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:06.456406 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:06.456376    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:06.469459 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:06.469422    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-wts6v"]
Jul 26 15:27:06.478022 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:06.477996    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wts6v"
Jul 26 15:27:06.478373 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:06.478349    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wts6v"]
Jul 26 15:27:06.480320 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:06.480288    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-chxxg\""
Jul 26 15:27:06.546782 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:06.546746    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk6w2\" (UniqueName: \"kubernetes.io/projected/fb9db141-18d2-4e2f-bcb0-29241d2ed4b6-kube-api-access-zk6w2\") pod \"authorino-75d556bd77-wts6v\" (UID: \"fb9db141-18d2-4e2f-bcb0-29241d2ed4b6\") " pod="kuadrant-system/authorino-75d556bd77-wts6v"
Jul 26 15:27:06.648299 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:06.648186    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk6w2\" (UniqueName: \"kubernetes.io/projected/fb9db141-18d2-4e2f-bcb0-29241d2ed4b6-kube-api-access-zk6w2\") pod \"authorino-75d556bd77-wts6v\" (UID: \"fb9db141-18d2-4e2f-bcb0-29241d2ed4b6\") " pod="kuadrant-system/authorino-75d556bd77-wts6v"
Jul 26 15:27:06.656693 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:06.656660    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk6w2\" (UniqueName: \"kubernetes.io/projected/fb9db141-18d2-4e2f-bcb0-29241d2ed4b6-kube-api-access-zk6w2\") pod \"authorino-75d556bd77-wts6v\" (UID: \"fb9db141-18d2-4e2f-bcb0-29241d2ed4b6\") " pod="kuadrant-system/authorino-75d556bd77-wts6v"
Jul 26 15:27:06.725868 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:06.725831    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwfsl" event={"ID":"081c1bc3-9b98-4342-b21d-d76c5d208bd5","Type":"ContainerStarted","Data":"f65ba0044728ffca02d61e2cd9a6fa7a29a8cd4be6eafa46a0f98cd053457e3c"}
Jul 26 15:27:06.791689 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:06.791422    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wts6v"
Jul 26 15:27:07.031728 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:07.031688    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wts6v"]
Jul 26 15:27:07.036737 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:27:07.036703    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb9db141_18d2_4e2f_bcb0_29241d2ed4b6.slice/crio-f6f3ac6865efd54be1690fcafc692881832d5d82fefc798dd2c6a19874b93a43 WatchSource:0}: Error finding container f6f3ac6865efd54be1690fcafc692881832d5d82fefc798dd2c6a19874b93a43: Status 404 returned error can't find the container with id f6f3ac6865efd54be1690fcafc692881832d5d82fefc798dd2c6a19874b93a43
Jul 26 15:27:07.732177 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:07.732138    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wts6v" event={"ID":"fb9db141-18d2-4e2f-bcb0-29241d2ed4b6","Type":"ContainerStarted","Data":"f6f3ac6865efd54be1690fcafc692881832d5d82fefc798dd2c6a19874b93a43"}
Jul 26 15:27:08.050044 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:08.049949    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:09.748002 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:09.747964    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwfsl" event={"ID":"081c1bc3-9b98-4342-b21d-d76c5d208bd5","Type":"ContainerStarted","Data":"b46826c4072b43becf4a7e0fe8bd611a8ac126561bfdcfbdecb2f22fdab59b00"}
Jul 26 15:27:09.748523 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:09.748096    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"
Jul 26 15:27:09.761790 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:09.761648    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwfsl" podStartSLOduration=1.848695478 podStartE2EDuration="4.761627393s" totalImagesPullingTime="2.912931915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:27:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:27:09.761056446 +0000 UTC m=+514.050888565" watchObservedRunningTime="2026-07-26 15:27:09.761627393 +0000 UTC m=+514.051459510"
Jul 26 15:27:11.760827 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:11.760788    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wts6v" event={"ID":"fb9db141-18d2-4e2f-bcb0-29241d2ed4b6","Type":"ContainerStarted","Data":"8b2168f26c0b2f075de55968cb838cff9540cb85e585f6d54b8b613590cafdda"}
Jul 26 15:27:11.775170 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:11.775114    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-wts6v" podStartSLOduration=1.8275058199999998 podStartE2EDuration="5.775097197s" totalImagesPullingTime="3.947591377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:27:11.774192662 +0000 UTC m=+516.064024782" watchObservedRunningTime="2026-07-26 15:27:11.775097197 +0000 UTC m=+516.064929315"
Jul 26 15:27:20.158696 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:20.158650    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"]
Jul 26 15:27:20.159137 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:20.158859    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwfsl" podUID="081c1bc3-9b98-4342-b21d-d76c5d208bd5" containerName="limitador" containerID="cri-o://b46826c4072b43becf4a7e0fe8bd611a8ac126561bfdcfbdecb2f22fdab59b00" gracePeriod=30
Jul 26 15:27:20.159607 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:20.159584    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"
Jul 26 15:27:20.709277 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:20.709248    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"
Jul 26 15:27:20.803773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:20.803686    2559 generic.go:356] "Generic (PLEG): container finished" podID="081c1bc3-9b98-4342-b21d-d76c5d208bd5" containerID="b46826c4072b43becf4a7e0fe8bd611a8ac126561bfdcfbdecb2f22fdab59b00" exitCode=0
Jul 26 15:27:20.803773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:20.803726    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwfsl" event={"ID":"081c1bc3-9b98-4342-b21d-d76c5d208bd5","Type":"ContainerDied","Data":"b46826c4072b43becf4a7e0fe8bd611a8ac126561bfdcfbdecb2f22fdab59b00"}
Jul 26 15:27:20.803773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:20.803729    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"
Jul 26 15:27:20.803773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:20.803769    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwfsl" event={"ID":"081c1bc3-9b98-4342-b21d-d76c5d208bd5","Type":"ContainerDied","Data":"f65ba0044728ffca02d61e2cd9a6fa7a29a8cd4be6eafa46a0f98cd053457e3c"}
Jul 26 15:27:20.804042 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:20.803791    2559 scope.go:117] "RemoveContainer" containerID="b46826c4072b43becf4a7e0fe8bd611a8ac126561bfdcfbdecb2f22fdab59b00"
Jul 26 15:27:20.812158 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:20.812137    2559 scope.go:117] "RemoveContainer" containerID="b46826c4072b43becf4a7e0fe8bd611a8ac126561bfdcfbdecb2f22fdab59b00"
Jul 26 15:27:20.812519 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:27:20.812493    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b46826c4072b43becf4a7e0fe8bd611a8ac126561bfdcfbdecb2f22fdab59b00\": container with ID starting with b46826c4072b43becf4a7e0fe8bd611a8ac126561bfdcfbdecb2f22fdab59b00 not found: ID does not exist" containerID="b46826c4072b43becf4a7e0fe8bd611a8ac126561bfdcfbdecb2f22fdab59b00"
Jul 26 15:27:20.812578 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:20.812529    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b46826c4072b43becf4a7e0fe8bd611a8ac126561bfdcfbdecb2f22fdab59b00"} err="failed to get container status \"b46826c4072b43becf4a7e0fe8bd611a8ac126561bfdcfbdecb2f22fdab59b00\": rpc error: code = NotFound desc = could not find container \"b46826c4072b43becf4a7e0fe8bd611a8ac126561bfdcfbdecb2f22fdab59b00\": container with ID starting with b46826c4072b43becf4a7e0fe8bd611a8ac126561bfdcfbdecb2f22fdab59b00 not found: ID does not exist"
Jul 26 15:27:20.879488 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:20.879450    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/081c1bc3-9b98-4342-b21d-d76c5d208bd5-config-file\") pod \"081c1bc3-9b98-4342-b21d-d76c5d208bd5\" (UID: \"081c1bc3-9b98-4342-b21d-d76c5d208bd5\") "
Jul 26 15:27:20.879614 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:20.879521    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94twh\" (UniqueName: \"kubernetes.io/projected/081c1bc3-9b98-4342-b21d-d76c5d208bd5-kube-api-access-94twh\") pod \"081c1bc3-9b98-4342-b21d-d76c5d208bd5\" (UID: \"081c1bc3-9b98-4342-b21d-d76c5d208bd5\") "
Jul 26 15:27:20.879811 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:20.879784    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/081c1bc3-9b98-4342-b21d-d76c5d208bd5-config-file" (OuterVolumeSpecName: "config-file") pod "081c1bc3-9b98-4342-b21d-d76c5d208bd5" (UID: "081c1bc3-9b98-4342-b21d-d76c5d208bd5"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:27:20.881556 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:20.881527    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/081c1bc3-9b98-4342-b21d-d76c5d208bd5-kube-api-access-94twh" (OuterVolumeSpecName: "kube-api-access-94twh") pod "081c1bc3-9b98-4342-b21d-d76c5d208bd5" (UID: "081c1bc3-9b98-4342-b21d-d76c5d208bd5"). InnerVolumeSpecName "kube-api-access-94twh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:27:20.981026 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:20.980974    2559 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/081c1bc3-9b98-4342-b21d-d76c5d208bd5-config-file\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:27:20.981026 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:20.981017    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94twh\" (UniqueName: \"kubernetes.io/projected/081c1bc3-9b98-4342-b21d-d76c5d208bd5-kube-api-access-94twh\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:27:21.130425 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:21.130348    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"]
Jul 26 15:27:21.131464 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:21.131437    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lwfsl"]
Jul 26 15:27:21.497411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:21.497284    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:21.596039 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:21.595990    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:21.948852 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:21.948812    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bq82x"]
Jul 26 15:27:21.949430 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:21.949409    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="081c1bc3-9b98-4342-b21d-d76c5d208bd5" containerName="limitador"
Jul 26 15:27:21.949430 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:21.949432    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="081c1bc3-9b98-4342-b21d-d76c5d208bd5" containerName="limitador"
Jul 26 15:27:21.949567 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:21.949556    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="081c1bc3-9b98-4342-b21d-d76c5d208bd5" containerName="limitador"
Jul 26 15:27:21.954141 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:21.954118    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bq82x"
Jul 26 15:27:21.956283 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:21.956250    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 26 15:27:21.956426 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:21.956298    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 15:27:21.956426 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:21.956313    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-bmtp7\""
Jul 26 15:27:21.956666 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:21.956645    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 26 15:27:21.961820 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:21.961773    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bq82x"]
Jul 26 15:27:21.977627 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:21.977593    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:22.091610 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:22.091573    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5zrz\" (UniqueName: \"kubernetes.io/projected/2bdd4663-f066-449f-ade8-ce0b25d36926-kube-api-access-c5zrz\") pod \"postgres-69fb4594cf-bq82x\" (UID: \"2bdd4663-f066-449f-ade8-ce0b25d36926\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bq82x"
Jul 26 15:27:22.091792 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:22.091622    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2bdd4663-f066-449f-ade8-ce0b25d36926-data\") pod \"postgres-69fb4594cf-bq82x\" (UID: \"2bdd4663-f066-449f-ade8-ce0b25d36926\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bq82x"
Jul 26 15:27:22.192238 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:22.192180    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5zrz\" (UniqueName: \"kubernetes.io/projected/2bdd4663-f066-449f-ade8-ce0b25d36926-kube-api-access-c5zrz\") pod \"postgres-69fb4594cf-bq82x\" (UID: \"2bdd4663-f066-449f-ade8-ce0b25d36926\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bq82x"
Jul 26 15:27:22.192422 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:22.192255    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2bdd4663-f066-449f-ade8-ce0b25d36926-data\") pod \"postgres-69fb4594cf-bq82x\" (UID: \"2bdd4663-f066-449f-ade8-ce0b25d36926\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bq82x"
Jul 26 15:27:22.192637 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:22.192618    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2bdd4663-f066-449f-ade8-ce0b25d36926-data\") pod \"postgres-69fb4594cf-bq82x\" (UID: \"2bdd4663-f066-449f-ade8-ce0b25d36926\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bq82x"
Jul 26 15:27:22.194392 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:22.194368    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="081c1bc3-9b98-4342-b21d-d76c5d208bd5" path="/var/lib/kubelet/pods/081c1bc3-9b98-4342-b21d-d76c5d208bd5/volumes"
Jul 26 15:27:22.200587 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:22.200516    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5zrz\" (UniqueName: \"kubernetes.io/projected/2bdd4663-f066-449f-ade8-ce0b25d36926-kube-api-access-c5zrz\") pod \"postgres-69fb4594cf-bq82x\" (UID: \"2bdd4663-f066-449f-ade8-ce0b25d36926\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bq82x"
Jul 26 15:27:22.266143 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:22.266104    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bq82x"
Jul 26 15:27:22.403256 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:22.403178    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bq82x"]
Jul 26 15:27:22.406276 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:27:22.406243    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bdd4663_f066_449f_ade8_ce0b25d36926.slice/crio-9b3c7e68db103e92903ad45ab46122edea66f4032bfe4bceb856f4cfb23d690b WatchSource:0}: Error finding container 9b3c7e68db103e92903ad45ab46122edea66f4032bfe4bceb856f4cfb23d690b: Status 404 returned error can't find the container with id 9b3c7e68db103e92903ad45ab46122edea66f4032bfe4bceb856f4cfb23d690b
Jul 26 15:27:22.815521 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:22.815477    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bq82x" event={"ID":"2bdd4663-f066-449f-ade8-ce0b25d36926","Type":"ContainerStarted","Data":"9b3c7e68db103e92903ad45ab46122edea66f4032bfe4bceb856f4cfb23d690b"}
Jul 26 15:27:27.847257 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:27.847155    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bq82x" event={"ID":"2bdd4663-f066-449f-ade8-ce0b25d36926","Type":"ContainerStarted","Data":"35d27f3aa6a801756dcc49947d78faaea3ed2ff742f68f641fe03fcd169b571b"}
Jul 26 15:27:27.847598 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:27.847274    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bq82x"
Jul 26 15:27:27.861907 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:27.861850    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bq82x" podStartSLOduration=2.121059504 podStartE2EDuration="6.861832752s" totalImagesPullingTime="4.740773248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:27:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:27:27.860483674 +0000 UTC m=+532.150315792" watchObservedRunningTime="2026-07-26 15:27:27.861832752 +0000 UTC m=+532.151664870"
Jul 26 15:27:33.879901 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:33.879869    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bq82x"
Jul 26 15:27:34.190049 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:34.189962    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:34.448941 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:34.448847    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:36.645172 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:36.645128    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wts6v"]
Jul 26 15:27:36.645784 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:36.645344    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-wts6v" podUID="fb9db141-18d2-4e2f-bcb0-29241d2ed4b6" containerName="authorino" containerID="cri-o://8b2168f26c0b2f075de55968cb838cff9540cb85e585f6d54b8b613590cafdda" gracePeriod=30
Jul 26 15:27:36.890172 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:36.890142    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb9db141-18d2-4e2f-bcb0-29241d2ed4b6" containerID="8b2168f26c0b2f075de55968cb838cff9540cb85e585f6d54b8b613590cafdda" exitCode=0
Jul 26 15:27:36.890297 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:36.890225    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wts6v" event={"ID":"fb9db141-18d2-4e2f-bcb0-29241d2ed4b6","Type":"ContainerDied","Data":"8b2168f26c0b2f075de55968cb838cff9540cb85e585f6d54b8b613590cafdda"}
Jul 26 15:27:36.890297 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:36.890269    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wts6v" event={"ID":"fb9db141-18d2-4e2f-bcb0-29241d2ed4b6","Type":"ContainerDied","Data":"f6f3ac6865efd54be1690fcafc692881832d5d82fefc798dd2c6a19874b93a43"}
Jul 26 15:27:36.890297 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:36.890281    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6f3ac6865efd54be1690fcafc692881832d5d82fefc798dd2c6a19874b93a43"
Jul 26 15:27:36.900228 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:36.900161    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wts6v"
Jul 26 15:27:36.924222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:36.924178    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zk6w2\" (UniqueName: \"kubernetes.io/projected/fb9db141-18d2-4e2f-bcb0-29241d2ed4b6-kube-api-access-zk6w2\") pod \"fb9db141-18d2-4e2f-bcb0-29241d2ed4b6\" (UID: \"fb9db141-18d2-4e2f-bcb0-29241d2ed4b6\") "
Jul 26 15:27:36.926212 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:36.926175    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb9db141-18d2-4e2f-bcb0-29241d2ed4b6-kube-api-access-zk6w2" (OuterVolumeSpecName: "kube-api-access-zk6w2") pod "fb9db141-18d2-4e2f-bcb0-29241d2ed4b6" (UID: "fb9db141-18d2-4e2f-bcb0-29241d2ed4b6"). InnerVolumeSpecName "kube-api-access-zk6w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:27:37.025182 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:37.025143    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zk6w2\" (UniqueName: \"kubernetes.io/projected/fb9db141-18d2-4e2f-bcb0-29241d2ed4b6-kube-api-access-zk6w2\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:27:37.895659 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:37.895627    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wts6v"
Jul 26 15:27:37.913978 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:37.913941    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wts6v"]
Jul 26 15:27:37.917815 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:37.917792    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wts6v"]
Jul 26 15:27:38.194216 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:38.194125    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb9db141-18d2-4e2f-bcb0-29241d2ed4b6" path="/var/lib/kubelet/pods/fb9db141-18d2-4e2f-bcb0-29241d2ed4b6/volumes"
Jul 26 15:27:41.166248 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.166193    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:41.181070 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.181037    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f864ff9b-rr9n4"]
Jul 26 15:27:41.181459 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.181444    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb9db141-18d2-4e2f-bcb0-29241d2ed4b6" containerName="authorino"
Jul 26 15:27:41.181516 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.181461    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb9db141-18d2-4e2f-bcb0-29241d2ed4b6" containerName="authorino"
Jul 26 15:27:41.181560 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.181533    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb9db141-18d2-4e2f-bcb0-29241d2ed4b6" containerName="authorino"
Jul 26 15:27:41.239529 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.239490    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f864ff9b-rr9n4"]
Jul 26 15:27:41.239708 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.239605    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f864ff9b-rr9n4"
Jul 26 15:27:41.242364 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.242341    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-j5ps2\""
Jul 26 15:27:41.242364 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.242360    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 26 15:27:41.242552 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.242366    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 26 15:27:41.252761 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.252736    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pxsv\" (UniqueName: \"kubernetes.io/projected/8c849f69-2082-4a56-80bf-ac5ded8c84eb-kube-api-access-7pxsv\") pod \"maas-controller-f864ff9b-rr9n4\" (UID: \"8c849f69-2082-4a56-80bf-ac5ded8c84eb\") " pod="opendatahub/maas-controller-f864ff9b-rr9n4"
Jul 26 15:27:41.252874 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.252788    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c849f69-2082-4a56-80bf-ac5ded8c84eb-webhook-cert\") pod \"maas-controller-f864ff9b-rr9n4\" (UID: \"8c849f69-2082-4a56-80bf-ac5ded8c84eb\") " pod="opendatahub/maas-controller-f864ff9b-rr9n4"
Jul 26 15:27:41.259410 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.259385    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:41.330079 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.330046    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:41.337217 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.337174    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84d47fc9db-jfnzv"]
Jul 26 15:27:41.353951 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.353920    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c849f69-2082-4a56-80bf-ac5ded8c84eb-webhook-cert\") pod \"maas-controller-f864ff9b-rr9n4\" (UID: \"8c849f69-2082-4a56-80bf-ac5ded8c84eb\") " pod="opendatahub/maas-controller-f864ff9b-rr9n4"
Jul 26 15:27:41.354126 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.353993    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pxsv\" (UniqueName: \"kubernetes.io/projected/8c849f69-2082-4a56-80bf-ac5ded8c84eb-kube-api-access-7pxsv\") pod \"maas-controller-f864ff9b-rr9n4\" (UID: \"8c849f69-2082-4a56-80bf-ac5ded8c84eb\") " pod="opendatahub/maas-controller-f864ff9b-rr9n4"
Jul 26 15:27:41.356419 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.356393    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c849f69-2082-4a56-80bf-ac5ded8c84eb-webhook-cert\") pod \"maas-controller-f864ff9b-rr9n4\" (UID: \"8c849f69-2082-4a56-80bf-ac5ded8c84eb\") " pod="opendatahub/maas-controller-f864ff9b-rr9n4"
Jul 26 15:27:41.358124 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.358097    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84d47fc9db-jfnzv"]
Jul 26 15:27:41.358251 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.358191    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d47fc9db-jfnzv"
Jul 26 15:27:41.361375 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.361352    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pxsv\" (UniqueName: \"kubernetes.io/projected/8c849f69-2082-4a56-80bf-ac5ded8c84eb-kube-api-access-7pxsv\") pod \"maas-controller-f864ff9b-rr9n4\" (UID: \"8c849f69-2082-4a56-80bf-ac5ded8c84eb\") " pod="opendatahub/maas-controller-f864ff9b-rr9n4"
Jul 26 15:27:41.454692 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.454602    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p8wn\" (UniqueName: \"kubernetes.io/projected/940fba82-f1d4-4365-a609-ac0fe91c50d4-kube-api-access-2p8wn\") pod \"maas-controller-84d47fc9db-jfnzv\" (UID: \"940fba82-f1d4-4365-a609-ac0fe91c50d4\") " pod="opendatahub/maas-controller-84d47fc9db-jfnzv"
Jul 26 15:27:41.454843 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.454701    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/940fba82-f1d4-4365-a609-ac0fe91c50d4-webhook-cert\") pod \"maas-controller-84d47fc9db-jfnzv\" (UID: \"940fba82-f1d4-4365-a609-ac0fe91c50d4\") " pod="opendatahub/maas-controller-84d47fc9db-jfnzv"
Jul 26 15:27:41.550803 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.550765    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f864ff9b-rr9n4"
Jul 26 15:27:41.555145 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.555115    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/940fba82-f1d4-4365-a609-ac0fe91c50d4-webhook-cert\") pod \"maas-controller-84d47fc9db-jfnzv\" (UID: \"940fba82-f1d4-4365-a609-ac0fe91c50d4\") " pod="opendatahub/maas-controller-84d47fc9db-jfnzv"
Jul 26 15:27:41.555276 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.555236    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p8wn\" (UniqueName: \"kubernetes.io/projected/940fba82-f1d4-4365-a609-ac0fe91c50d4-kube-api-access-2p8wn\") pod \"maas-controller-84d47fc9db-jfnzv\" (UID: \"940fba82-f1d4-4365-a609-ac0fe91c50d4\") " pod="opendatahub/maas-controller-84d47fc9db-jfnzv"
Jul 26 15:27:41.557449 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.557423    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/940fba82-f1d4-4365-a609-ac0fe91c50d4-webhook-cert\") pod \"maas-controller-84d47fc9db-jfnzv\" (UID: \"940fba82-f1d4-4365-a609-ac0fe91c50d4\") " pod="opendatahub/maas-controller-84d47fc9db-jfnzv"
Jul 26 15:27:41.562362 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.562339    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p8wn\" (UniqueName: \"kubernetes.io/projected/940fba82-f1d4-4365-a609-ac0fe91c50d4-kube-api-access-2p8wn\") pod \"maas-controller-84d47fc9db-jfnzv\" (UID: \"940fba82-f1d4-4365-a609-ac0fe91c50d4\") " pod="opendatahub/maas-controller-84d47fc9db-jfnzv"
Jul 26 15:27:41.681446 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.681411    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d47fc9db-jfnzv"
Jul 26 15:27:41.691410 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.691379    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f864ff9b-rr9n4"]
Jul 26 15:27:41.693603 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:27:41.693572    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c849f69_2082_4a56_80bf_ac5ded8c84eb.slice/crio-1f2515ebaf2909b0af649b65a620ef29a31212bdee18ce1ab65a858308fc3cd4 WatchSource:0}: Error finding container 1f2515ebaf2909b0af649b65a620ef29a31212bdee18ce1ab65a858308fc3cd4: Status 404 returned error can't find the container with id 1f2515ebaf2909b0af649b65a620ef29a31212bdee18ce1ab65a858308fc3cd4
Jul 26 15:27:41.821756 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.821735    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84d47fc9db-jfnzv"]
Jul 26 15:27:41.823829 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:27:41.823800    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod940fba82_f1d4_4365_a609_ac0fe91c50d4.slice/crio-bcffa439dd8ca6209f0318d50828924c92931abd23b0fc7b25d131788591360a WatchSource:0}: Error finding container bcffa439dd8ca6209f0318d50828924c92931abd23b0fc7b25d131788591360a: Status 404 returned error can't find the container with id bcffa439dd8ca6209f0318d50828924c92931abd23b0fc7b25d131788591360a
Jul 26 15:27:41.915001 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.914960    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d47fc9db-jfnzv" event={"ID":"940fba82-f1d4-4365-a609-ac0fe91c50d4","Type":"ContainerStarted","Data":"bcffa439dd8ca6209f0318d50828924c92931abd23b0fc7b25d131788591360a"}
Jul 26 15:27:41.916005 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:41.915985    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f864ff9b-rr9n4" event={"ID":"8c849f69-2082-4a56-80bf-ac5ded8c84eb","Type":"ContainerStarted","Data":"1f2515ebaf2909b0af649b65a620ef29a31212bdee18ce1ab65a858308fc3cd4"}
Jul 26 15:27:44.478696 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:44.478664    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:44.572620 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:44.572588    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:44.937637 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:44.937542    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d47fc9db-jfnzv" event={"ID":"940fba82-f1d4-4365-a609-ac0fe91c50d4","Type":"ContainerStarted","Data":"c3d49840c889337d82f827c877be6ee27a676d06118ab06629e38203bf0e7794"}
Jul 26 15:27:44.937784 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:44.937645    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84d47fc9db-jfnzv"
Jul 26 15:27:44.939215 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:44.939177    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f864ff9b-rr9n4" event={"ID":"8c849f69-2082-4a56-80bf-ac5ded8c84eb","Type":"ContainerStarted","Data":"ff61dd09f28a0067c6faae4089e409939e4e4f696703069945cbeb1d2120bfe9"}
Jul 26 15:27:44.939358 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:44.939249    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f864ff9b-rr9n4"
Jul 26 15:27:44.954387 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:44.954339    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-84d47fc9db-jfnzv" podStartSLOduration=1.491575554 podStartE2EDuration="3.954325066s" totalImagesPullingTime="2.462749512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:27:44.953008208 +0000 UTC m=+549.242840320" watchObservedRunningTime="2026-07-26 15:27:44.954325066 +0000 UTC m=+549.244157184"
Jul 26 15:27:44.970662 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:44.970605    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-f864ff9b-rr9n4" podStartSLOduration=1.3819442880000001 podStartE2EDuration="3.970586698s" totalImagesPullingTime="2.58864241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:27:44.969159878 +0000 UTC m=+549.258991997" watchObservedRunningTime="2026-07-26 15:27:44.970586698 +0000 UTC m=+549.260418863"
Jul 26 15:27:50.281546 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:50.281509    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:50.381526 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:50.381489    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:27:55.946261 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:55.946229    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f864ff9b-rr9n4"
Jul 26 15:27:55.946671 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:55.946302    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84d47fc9db-jfnzv"
Jul 26 15:27:56.001212 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:56.001167    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f864ff9b-rr9n4"]
Jul 26 15:27:56.001525 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:56.001488    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-f864ff9b-rr9n4" podUID="8c849f69-2082-4a56-80bf-ac5ded8c84eb" containerName="manager" containerID="cri-o://ff61dd09f28a0067c6faae4089e409939e4e4f696703069945cbeb1d2120bfe9" gracePeriod=30
Jul 26 15:27:56.242901 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:56.242877    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f864ff9b-rr9n4"
Jul 26 15:27:56.287988 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:56.287955    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pxsv\" (UniqueName: \"kubernetes.io/projected/8c849f69-2082-4a56-80bf-ac5ded8c84eb-kube-api-access-7pxsv\") pod \"8c849f69-2082-4a56-80bf-ac5ded8c84eb\" (UID: \"8c849f69-2082-4a56-80bf-ac5ded8c84eb\") "
Jul 26 15:27:56.288167 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:56.288049    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c849f69-2082-4a56-80bf-ac5ded8c84eb-webhook-cert\") pod \"8c849f69-2082-4a56-80bf-ac5ded8c84eb\" (UID: \"8c849f69-2082-4a56-80bf-ac5ded8c84eb\") "
Jul 26 15:27:56.290473 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:56.290443    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c849f69-2082-4a56-80bf-ac5ded8c84eb-kube-api-access-7pxsv" (OuterVolumeSpecName: "kube-api-access-7pxsv") pod "8c849f69-2082-4a56-80bf-ac5ded8c84eb" (UID: "8c849f69-2082-4a56-80bf-ac5ded8c84eb"). InnerVolumeSpecName "kube-api-access-7pxsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:27:56.290473 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:56.290456    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c849f69-2082-4a56-80bf-ac5ded8c84eb-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8c849f69-2082-4a56-80bf-ac5ded8c84eb" (UID: "8c849f69-2082-4a56-80bf-ac5ded8c84eb"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:27:56.389630 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:56.389590    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pxsv\" (UniqueName: \"kubernetes.io/projected/8c849f69-2082-4a56-80bf-ac5ded8c84eb-kube-api-access-7pxsv\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:27:56.389630 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:56.389624    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c849f69-2082-4a56-80bf-ac5ded8c84eb-webhook-cert\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:27:56.997186 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:56.997151    2559 generic.go:356] "Generic (PLEG): container finished" podID="8c849f69-2082-4a56-80bf-ac5ded8c84eb" containerID="ff61dd09f28a0067c6faae4089e409939e4e4f696703069945cbeb1d2120bfe9" exitCode=0
Jul 26 15:27:56.997627 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:56.997225    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f864ff9b-rr9n4"
Jul 26 15:27:56.997627 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:56.997244    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f864ff9b-rr9n4" event={"ID":"8c849f69-2082-4a56-80bf-ac5ded8c84eb","Type":"ContainerDied","Data":"ff61dd09f28a0067c6faae4089e409939e4e4f696703069945cbeb1d2120bfe9"}
Jul 26 15:27:56.997627 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:56.997319    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f864ff9b-rr9n4" event={"ID":"8c849f69-2082-4a56-80bf-ac5ded8c84eb","Type":"ContainerDied","Data":"1f2515ebaf2909b0af649b65a620ef29a31212bdee18ce1ab65a858308fc3cd4"}
Jul 26 15:27:56.997627 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:56.997342    2559 scope.go:117] "RemoveContainer" containerID="ff61dd09f28a0067c6faae4089e409939e4e4f696703069945cbeb1d2120bfe9"
Jul 26 15:27:57.006384 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:57.006115    2559 scope.go:117] "RemoveContainer" containerID="ff61dd09f28a0067c6faae4089e409939e4e4f696703069945cbeb1d2120bfe9"
Jul 26 15:27:57.006439 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:27:57.006393    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff61dd09f28a0067c6faae4089e409939e4e4f696703069945cbeb1d2120bfe9\": container with ID starting with ff61dd09f28a0067c6faae4089e409939e4e4f696703069945cbeb1d2120bfe9 not found: ID does not exist" containerID="ff61dd09f28a0067c6faae4089e409939e4e4f696703069945cbeb1d2120bfe9"
Jul 26 15:27:57.006439 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:57.006419    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff61dd09f28a0067c6faae4089e409939e4e4f696703069945cbeb1d2120bfe9"} err="failed to get container status \"ff61dd09f28a0067c6faae4089e409939e4e4f696703069945cbeb1d2120bfe9\": rpc error: code = NotFound desc = could not find container \"ff61dd09f28a0067c6faae4089e409939e4e4f696703069945cbeb1d2120bfe9\": container with ID starting with ff61dd09f28a0067c6faae4089e409939e4e4f696703069945cbeb1d2120bfe9 not found: ID does not exist"
Jul 26 15:27:57.016508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:57.016484    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f864ff9b-rr9n4"]
Jul 26 15:27:57.022122 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:57.022101    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-f864ff9b-rr9n4"]
Jul 26 15:27:58.193245 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:27:58.193186    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c849f69-2082-4a56-80bf-ac5ded8c84eb" path="/var/lib/kubelet/pods/8c849f69-2082-4a56-80bf-ac5ded8c84eb/volumes"
Jul 26 15:28:16.085253 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:16.085216    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:28:16.179684 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:16.179649    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:28:16.732734 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:16.732702    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:28:16.786393 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:16.786360    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6978d97978-hh2xg"]
Jul 26 15:28:16.786729 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:16.786717    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c849f69-2082-4a56-80bf-ac5ded8c84eb" containerName="manager"
Jul 26 15:28:16.786775 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:16.786730    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c849f69-2082-4a56-80bf-ac5ded8c84eb" containerName="manager"
Jul 26 15:28:16.786810 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:16.786805    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c849f69-2082-4a56-80bf-ac5ded8c84eb" containerName="manager"
Jul 26 15:28:16.795462 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:16.795431    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6978d97978-hh2xg"
Jul 26 15:28:16.797516 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:16.797486    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 26 15:28:16.797720 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:16.797688    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4lrpb\""
Jul 26 15:28:16.798687 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:16.798661    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6978d97978-hh2xg"]
Jul 26 15:28:16.969255 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:16.969190    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6cvl\" (UniqueName: \"kubernetes.io/projected/c1c3b4cc-199c-4612-9b7d-9ff5273b7efe-kube-api-access-x6cvl\") pod \"maas-api-6978d97978-hh2xg\" (UID: \"c1c3b4cc-199c-4612-9b7d-9ff5273b7efe\") " pod="odh-ai-gateway-infra/maas-api-6978d97978-hh2xg"
Jul 26 15:28:16.969434 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:16.969322    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1c3b4cc-199c-4612-9b7d-9ff5273b7efe-maas-api-tls\") pod \"maas-api-6978d97978-hh2xg\" (UID: \"c1c3b4cc-199c-4612-9b7d-9ff5273b7efe\") " pod="odh-ai-gateway-infra/maas-api-6978d97978-hh2xg"
Jul 26 15:28:16.969434 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:16.969355    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1c3b4cc-199c-4612-9b7d-9ff5273b7efe-openshift-service-ca\") pod \"maas-api-6978d97978-hh2xg\" (UID: \"c1c3b4cc-199c-4612-9b7d-9ff5273b7efe\") " pod="odh-ai-gateway-infra/maas-api-6978d97978-hh2xg"
Jul 26 15:28:16.981285 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:16.981254    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:28:17.070639 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.070565    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1c3b4cc-199c-4612-9b7d-9ff5273b7efe-maas-api-tls\") pod \"maas-api-6978d97978-hh2xg\" (UID: \"c1c3b4cc-199c-4612-9b7d-9ff5273b7efe\") " pod="odh-ai-gateway-infra/maas-api-6978d97978-hh2xg"
Jul 26 15:28:17.070639 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.070643    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1c3b4cc-199c-4612-9b7d-9ff5273b7efe-openshift-service-ca\") pod \"maas-api-6978d97978-hh2xg\" (UID: \"c1c3b4cc-199c-4612-9b7d-9ff5273b7efe\") " pod="odh-ai-gateway-infra/maas-api-6978d97978-hh2xg"
Jul 26 15:28:17.070899 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.070745    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6cvl\" (UniqueName: \"kubernetes.io/projected/c1c3b4cc-199c-4612-9b7d-9ff5273b7efe-kube-api-access-x6cvl\") pod \"maas-api-6978d97978-hh2xg\" (UID: \"c1c3b4cc-199c-4612-9b7d-9ff5273b7efe\") " pod="odh-ai-gateway-infra/maas-api-6978d97978-hh2xg"
Jul 26 15:28:17.071486 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.071462    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1c3b4cc-199c-4612-9b7d-9ff5273b7efe-openshift-service-ca\") pod \"maas-api-6978d97978-hh2xg\" (UID: \"c1c3b4cc-199c-4612-9b7d-9ff5273b7efe\") " pod="odh-ai-gateway-infra/maas-api-6978d97978-hh2xg"
Jul 26 15:28:17.073577 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.073548    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1c3b4cc-199c-4612-9b7d-9ff5273b7efe-maas-api-tls\") pod \"maas-api-6978d97978-hh2xg\" (UID: \"c1c3b4cc-199c-4612-9b7d-9ff5273b7efe\") " pod="odh-ai-gateway-infra/maas-api-6978d97978-hh2xg"
Jul 26 15:28:17.078289 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.078261    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6cvl\" (UniqueName: \"kubernetes.io/projected/c1c3b4cc-199c-4612-9b7d-9ff5273b7efe-kube-api-access-x6cvl\") pod \"maas-api-6978d97978-hh2xg\" (UID: \"c1c3b4cc-199c-4612-9b7d-9ff5273b7efe\") " pod="odh-ai-gateway-infra/maas-api-6978d97978-hh2xg"
Jul 26 15:28:17.107867 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.107833    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6978d97978-hh2xg"
Jul 26 15:28:17.252500 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.252471    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:28:17.260350 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.260325    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6978d97978-hh2xg"]
Jul 26 15:28:17.262418 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:28:17.262391    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1c3b4cc_199c_4612_9b7d_9ff5273b7efe.slice/crio-cfc45001e5691aeca876bd30b27969ebfc959df3e8e7c8b32b9fc40417009ee6 WatchSource:0}: Error finding container cfc45001e5691aeca876bd30b27969ebfc959df3e8e7c8b32b9fc40417009ee6: Status 404 returned error can't find the container with id cfc45001e5691aeca876bd30b27969ebfc959df3e8e7c8b32b9fc40417009ee6
Jul 26 15:28:17.358001 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.357913    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:28:17.365472 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.365442    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-wsp9v"]
Jul 26 15:28:17.371375 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.371349    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-vc5dp"]
Jul 26 15:28:17.371522 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.371464    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-wsp9v"
Jul 26 15:28:17.373603 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.373480    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 26 15:28:17.373749 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.373624    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-v6955\""
Jul 26 15:28:17.376422 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.376402    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vc5dp"
Jul 26 15:28:17.382751 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.381638    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-wsp9v"]
Jul 26 15:28:17.383781 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.383739    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-vc5dp"]
Jul 26 15:28:17.474059 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.474023    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vzfc\" (UniqueName: \"kubernetes.io/projected/afc1d831-651b-4136-8a8e-8440f9a966c3-kube-api-access-7vzfc\") pod \"payload-processing-5cc6cd444b-wsp9v\" (UID: \"afc1d831-651b-4136-8a8e-8440f9a966c3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wsp9v"
Jul 26 15:28:17.474059 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.474069    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2826\" (UniqueName: \"kubernetes.io/projected/58a004c8-8db2-46f4-851a-fa381aa8024a-kube-api-access-s2826\") pod \"payload-pre-processing-687dbb6cf6-vc5dp\" (UID: \"58a004c8-8db2-46f4-851a-fa381aa8024a\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vc5dp"
Jul 26 15:28:17.474332 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.474094    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58a004c8-8db2-46f4-851a-fa381aa8024a-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-vc5dp\" (UID: \"58a004c8-8db2-46f4-851a-fa381aa8024a\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vc5dp"
Jul 26 15:28:17.474332 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.474235    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afc1d831-651b-4136-8a8e-8440f9a966c3-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-wsp9v\" (UID: \"afc1d831-651b-4136-8a8e-8440f9a966c3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wsp9v"
Jul 26 15:28:17.575018 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.574976    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vzfc\" (UniqueName: \"kubernetes.io/projected/afc1d831-651b-4136-8a8e-8440f9a966c3-kube-api-access-7vzfc\") pod \"payload-processing-5cc6cd444b-wsp9v\" (UID: \"afc1d831-651b-4136-8a8e-8440f9a966c3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wsp9v"
Jul 26 15:28:17.575260 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.575032    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2826\" (UniqueName: \"kubernetes.io/projected/58a004c8-8db2-46f4-851a-fa381aa8024a-kube-api-access-s2826\") pod \"payload-pre-processing-687dbb6cf6-vc5dp\" (UID: \"58a004c8-8db2-46f4-851a-fa381aa8024a\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vc5dp"
Jul 26 15:28:17.575260 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.575063    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58a004c8-8db2-46f4-851a-fa381aa8024a-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-vc5dp\" (UID: \"58a004c8-8db2-46f4-851a-fa381aa8024a\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vc5dp"
Jul 26 15:28:17.575260 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.575124    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afc1d831-651b-4136-8a8e-8440f9a966c3-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-wsp9v\" (UID: \"afc1d831-651b-4136-8a8e-8440f9a966c3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wsp9v"
Jul 26 15:28:17.575862 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.575836    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58a004c8-8db2-46f4-851a-fa381aa8024a-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-vc5dp\" (UID: \"58a004c8-8db2-46f4-851a-fa381aa8024a\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vc5dp"
Jul 26 15:28:17.575974 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.575959    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afc1d831-651b-4136-8a8e-8440f9a966c3-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-wsp9v\" (UID: \"afc1d831-651b-4136-8a8e-8440f9a966c3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wsp9v"
Jul 26 15:28:17.582609 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.582581    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vzfc\" (UniqueName: \"kubernetes.io/projected/afc1d831-651b-4136-8a8e-8440f9a966c3-kube-api-access-7vzfc\") pod \"payload-processing-5cc6cd444b-wsp9v\" (UID: \"afc1d831-651b-4136-8a8e-8440f9a966c3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wsp9v"
Jul 26 15:28:17.582707 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.582628    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2826\" (UniqueName: \"kubernetes.io/projected/58a004c8-8db2-46f4-851a-fa381aa8024a-kube-api-access-s2826\") pod \"payload-pre-processing-687dbb6cf6-vc5dp\" (UID: \"58a004c8-8db2-46f4-851a-fa381aa8024a\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vc5dp"
Jul 26 15:28:17.687372 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.687283    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-wsp9v"
Jul 26 15:28:17.695183 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.695153    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vc5dp"
Jul 26 15:28:17.834526 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.834486    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-wsp9v"]
Jul 26 15:28:17.838531 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:28:17.838491    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafc1d831_651b_4136_8a8e_8440f9a966c3.slice/crio-752b6c99c1cf1b00cee64894cd540aa979f50899ba5f2e2a28fccc41c0986da7 WatchSource:0}: Error finding container 752b6c99c1cf1b00cee64894cd540aa979f50899ba5f2e2a28fccc41c0986da7: Status 404 returned error can't find the container with id 752b6c99c1cf1b00cee64894cd540aa979f50899ba5f2e2a28fccc41c0986da7
Jul 26 15:28:17.855938 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:17.855907    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-vc5dp"]
Jul 26 15:28:17.858828 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:28:17.858802    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58a004c8_8db2_46f4_851a_fa381aa8024a.slice/crio-0e3753ec6d9ebdb786049007257d593bffe15baabd0f1d1828a12e3e53ae6f84 WatchSource:0}: Error finding container 0e3753ec6d9ebdb786049007257d593bffe15baabd0f1d1828a12e3e53ae6f84: Status 404 returned error can't find the container with id 0e3753ec6d9ebdb786049007257d593bffe15baabd0f1d1828a12e3e53ae6f84
Jul 26 15:28:18.100949 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:18.100889    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-wsp9v" event={"ID":"afc1d831-651b-4136-8a8e-8440f9a966c3","Type":"ContainerStarted","Data":"752b6c99c1cf1b00cee64894cd540aa979f50899ba5f2e2a28fccc41c0986da7"}
Jul 26 15:28:18.102213 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:18.102177    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6978d97978-hh2xg" event={"ID":"c1c3b4cc-199c-4612-9b7d-9ff5273b7efe","Type":"ContainerStarted","Data":"cfc45001e5691aeca876bd30b27969ebfc959df3e8e7c8b32b9fc40417009ee6"}
Jul 26 15:28:18.103453 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:18.103427    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vc5dp" event={"ID":"58a004c8-8db2-46f4-851a-fa381aa8024a","Type":"ContainerStarted","Data":"0e3753ec6d9ebdb786049007257d593bffe15baabd0f1d1828a12e3e53ae6f84"}
Jul 26 15:28:20.118473 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:20.118381    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6978d97978-hh2xg" event={"ID":"c1c3b4cc-199c-4612-9b7d-9ff5273b7efe","Type":"ContainerStarted","Data":"d7a02af59dc2855cc7daa0b8e2df2a2549427710455785f5160eab0c59cf2834"}
Jul 26 15:28:20.118473 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:20.118472    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6978d97978-hh2xg"
Jul 26 15:28:20.119928 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:20.119905    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vc5dp" event={"ID":"58a004c8-8db2-46f4-851a-fa381aa8024a","Type":"ContainerStarted","Data":"67bb8c0a3bf465a2f766a949bd3acda8f947a32c6be004630d81ba04f261f34a"}
Jul 26 15:28:20.120040 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:20.119973    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vc5dp"
Jul 26 15:28:20.121293 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:20.121266    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-wsp9v" event={"ID":"afc1d831-651b-4136-8a8e-8440f9a966c3","Type":"ContainerStarted","Data":"00de9738385d451ecf0d2f67ee1a54076c95f62cfb240f302250054feb7cc7d6"}
Jul 26 15:28:20.121410 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:20.121396    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-wsp9v"
Jul 26 15:28:20.133938 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:20.133888    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6978d97978-hh2xg" podStartSLOduration=2.105075677 podStartE2EDuration="4.133871686s" totalImagesPullingTime="2.028796009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:28:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:28:20.133310591 +0000 UTC m=+584.423142707" watchObservedRunningTime="2026-07-26 15:28:20.133871686 +0000 UTC m=+584.423703806"
Jul 26 15:28:20.147866 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:20.147822    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vc5dp" podStartSLOduration=1.181990878 podStartE2EDuration="3.147807211s" totalImagesPullingTime="1.965816333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:28:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:28:20.146710529 +0000 UTC m=+584.436542649" watchObservedRunningTime="2026-07-26 15:28:20.147807211 +0000 UTC m=+584.437639308"
Jul 26 15:28:20.161482 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:20.161412    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-wsp9v" podStartSLOduration=1.181276421 podStartE2EDuration="3.161393901s" totalImagesPullingTime="1.98011748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:28:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:28:20.159732493 +0000 UTC m=+584.449564614" watchObservedRunningTime="2026-07-26 15:28:20.161393901 +0000 UTC m=+584.451226025"
Jul 26 15:28:26.131932 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:26.131899    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6978d97978-hh2xg"
Jul 26 15:28:27.731052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:27.731015    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:28:28.845492 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:28.845459    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:28:31.128723 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:31.128687    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-wsp9v"
Jul 26 15:28:31.129149 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:31.128861    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vc5dp"
Jul 26 15:28:32.694437 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.694406    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"]
Jul 26 15:28:32.697847 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.697827    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:32.699852 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.699828    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 26 15:28:32.699973 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.699865    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 26 15:28:32.700279 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.700263    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-c5r48\""
Jul 26 15:28:32.700332 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.700273    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 26 15:28:32.707030 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.707001    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"]
Jul 26 15:28:32.720047 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.720021    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:28:32.721163 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.721139    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7m7v\" (UniqueName: \"kubernetes.io/projected/1c73a48a-0e06-4de7-bd2a-9fb758ab4289-kube-api-access-n7m7v\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x\" (UID: \"1c73a48a-0e06-4de7-bd2a-9fb758ab4289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:32.721288 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.721181    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c73a48a-0e06-4de7-bd2a-9fb758ab4289-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x\" (UID: \"1c73a48a-0e06-4de7-bd2a-9fb758ab4289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:32.721288 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.721214    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c73a48a-0e06-4de7-bd2a-9fb758ab4289-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x\" (UID: \"1c73a48a-0e06-4de7-bd2a-9fb758ab4289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:32.721288 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.721262    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c73a48a-0e06-4de7-bd2a-9fb758ab4289-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x\" (UID: \"1c73a48a-0e06-4de7-bd2a-9fb758ab4289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:32.721288 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.721286    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c73a48a-0e06-4de7-bd2a-9fb758ab4289-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x\" (UID: \"1c73a48a-0e06-4de7-bd2a-9fb758ab4289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:32.721502 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.721332    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c73a48a-0e06-4de7-bd2a-9fb758ab4289-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x\" (UID: \"1c73a48a-0e06-4de7-bd2a-9fb758ab4289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:32.821960 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.821923    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7m7v\" (UniqueName: \"kubernetes.io/projected/1c73a48a-0e06-4de7-bd2a-9fb758ab4289-kube-api-access-n7m7v\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x\" (UID: \"1c73a48a-0e06-4de7-bd2a-9fb758ab4289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:32.822146 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.821972    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c73a48a-0e06-4de7-bd2a-9fb758ab4289-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x\" (UID: \"1c73a48a-0e06-4de7-bd2a-9fb758ab4289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:32.822146 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.821999    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c73a48a-0e06-4de7-bd2a-9fb758ab4289-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x\" (UID: \"1c73a48a-0e06-4de7-bd2a-9fb758ab4289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:32.822146 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.822029    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c73a48a-0e06-4de7-bd2a-9fb758ab4289-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x\" (UID: \"1c73a48a-0e06-4de7-bd2a-9fb758ab4289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:32.822146 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.822058    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c73a48a-0e06-4de7-bd2a-9fb758ab4289-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x\" (UID: \"1c73a48a-0e06-4de7-bd2a-9fb758ab4289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:32.822146 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.822097    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c73a48a-0e06-4de7-bd2a-9fb758ab4289-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x\" (UID: \"1c73a48a-0e06-4de7-bd2a-9fb758ab4289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:32.822462 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.822429    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c73a48a-0e06-4de7-bd2a-9fb758ab4289-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x\" (UID: \"1c73a48a-0e06-4de7-bd2a-9fb758ab4289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:32.822462 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.822452    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c73a48a-0e06-4de7-bd2a-9fb758ab4289-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x\" (UID: \"1c73a48a-0e06-4de7-bd2a-9fb758ab4289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:32.822567 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.822515    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c73a48a-0e06-4de7-bd2a-9fb758ab4289-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x\" (UID: \"1c73a48a-0e06-4de7-bd2a-9fb758ab4289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:32.824346 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.824320    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c73a48a-0e06-4de7-bd2a-9fb758ab4289-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x\" (UID: \"1c73a48a-0e06-4de7-bd2a-9fb758ab4289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:32.824608 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.824591    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c73a48a-0e06-4de7-bd2a-9fb758ab4289-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x\" (UID: \"1c73a48a-0e06-4de7-bd2a-9fb758ab4289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:32.829901 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:32.829875    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7m7v\" (UniqueName: \"kubernetes.io/projected/1c73a48a-0e06-4de7-bd2a-9fb758ab4289-kube-api-access-n7m7v\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x\" (UID: \"1c73a48a-0e06-4de7-bd2a-9fb758ab4289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:33.009010 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:33.008927    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:33.139691 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:33.139665    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"]
Jul 26 15:28:33.142118 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:28:33.142090    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c73a48a_0e06_4de7_bd2a_9fb758ab4289.slice/crio-66ca29f73289a18bda723003da02eadc10376bf027b6052e9d6f5bec34a544c6 WatchSource:0}: Error finding container 66ca29f73289a18bda723003da02eadc10376bf027b6052e9d6f5bec34a544c6: Status 404 returned error can't find the container with id 66ca29f73289a18bda723003da02eadc10376bf027b6052e9d6f5bec34a544c6
Jul 26 15:28:33.185784 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:33.185751    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x" event={"ID":"1c73a48a-0e06-4de7-bd2a-9fb758ab4289","Type":"ContainerStarted","Data":"66ca29f73289a18bda723003da02eadc10376bf027b6052e9d6f5bec34a544c6"}
Jul 26 15:28:33.541561 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:33.541509    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:28:35.196221 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:35.196167    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x" event={"ID":"1c73a48a-0e06-4de7-bd2a-9fb758ab4289","Type":"ContainerStarted","Data":"dd9231fb81a03de5db2991cf02b24d55589a81e2050a29ecbe5f0d048a41e268"}
Jul 26 15:28:35.212997 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:35.212949    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x" podStartSLOduration=2.131159292 podStartE2EDuration="3.212933796s" totalImagesPullingTime="1.081774504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:28:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:28:35.21124124 +0000 UTC m=+599.501073358" watchObservedRunningTime="2026-07-26 15:28:35.212933796 +0000 UTC m=+599.502765913"
Jul 26 15:28:37.112687 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:37.112640    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:28:43.009911 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:43.009876    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:43.012360 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:43.009948    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:43.022326 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:43.022304    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:28:43.245484 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:28:43.245458    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x"
Jul 26 15:29:00.709910 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:00.709833    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:29:24.477065 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:24.477028    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:29:37.285833 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.285794    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"]
Jul 26 15:29:37.289427 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.289406    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.291519 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.291492    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 26 15:29:37.299298 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.299274    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"]
Jul 26 15:29:37.315144 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.315103    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:29:37.412697 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.412662    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a47137a-5f31-4db0-ada4-1f9f3741c618-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qm6ms\" (UID: \"7a47137a-5f31-4db0-ada4-1f9f3741c618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.412863 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.412713    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a47137a-5f31-4db0-ada4-1f9f3741c618-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qm6ms\" (UID: \"7a47137a-5f31-4db0-ada4-1f9f3741c618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.412863 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.412733    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a47137a-5f31-4db0-ada4-1f9f3741c618-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qm6ms\" (UID: \"7a47137a-5f31-4db0-ada4-1f9f3741c618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.412863 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.412793    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a47137a-5f31-4db0-ada4-1f9f3741c618-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qm6ms\" (UID: \"7a47137a-5f31-4db0-ada4-1f9f3741c618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.412967 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.412869    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a47137a-5f31-4db0-ada4-1f9f3741c618-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qm6ms\" (UID: \"7a47137a-5f31-4db0-ada4-1f9f3741c618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.412967 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.412896    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5sct\" (UniqueName: \"kubernetes.io/projected/7a47137a-5f31-4db0-ada4-1f9f3741c618-kube-api-access-x5sct\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qm6ms\" (UID: \"7a47137a-5f31-4db0-ada4-1f9f3741c618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.513755 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.513718    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a47137a-5f31-4db0-ada4-1f9f3741c618-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qm6ms\" (UID: \"7a47137a-5f31-4db0-ada4-1f9f3741c618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.513941 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.513768    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a47137a-5f31-4db0-ada4-1f9f3741c618-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qm6ms\" (UID: \"7a47137a-5f31-4db0-ada4-1f9f3741c618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.513941 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.513789    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a47137a-5f31-4db0-ada4-1f9f3741c618-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qm6ms\" (UID: \"7a47137a-5f31-4db0-ada4-1f9f3741c618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.513941 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.513810    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a47137a-5f31-4db0-ada4-1f9f3741c618-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qm6ms\" (UID: \"7a47137a-5f31-4db0-ada4-1f9f3741c618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.513941 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.513849    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a47137a-5f31-4db0-ada4-1f9f3741c618-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qm6ms\" (UID: \"7a47137a-5f31-4db0-ada4-1f9f3741c618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.513941 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.513871    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5sct\" (UniqueName: \"kubernetes.io/projected/7a47137a-5f31-4db0-ada4-1f9f3741c618-kube-api-access-x5sct\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qm6ms\" (UID: \"7a47137a-5f31-4db0-ada4-1f9f3741c618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.514292 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.514262    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a47137a-5f31-4db0-ada4-1f9f3741c618-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qm6ms\" (UID: \"7a47137a-5f31-4db0-ada4-1f9f3741c618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.514486 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.514289    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a47137a-5f31-4db0-ada4-1f9f3741c618-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qm6ms\" (UID: \"7a47137a-5f31-4db0-ada4-1f9f3741c618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.514486 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.514316    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a47137a-5f31-4db0-ada4-1f9f3741c618-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qm6ms\" (UID: \"7a47137a-5f31-4db0-ada4-1f9f3741c618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.516022 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.515986    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a47137a-5f31-4db0-ada4-1f9f3741c618-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qm6ms\" (UID: \"7a47137a-5f31-4db0-ada4-1f9f3741c618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.516322 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.516307    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a47137a-5f31-4db0-ada4-1f9f3741c618-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qm6ms\" (UID: \"7a47137a-5f31-4db0-ada4-1f9f3741c618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.522262 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.522198    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5sct\" (UniqueName: \"kubernetes.io/projected/7a47137a-5f31-4db0-ada4-1f9f3741c618-kube-api-access-x5sct\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qm6ms\" (UID: \"7a47137a-5f31-4db0-ada4-1f9f3741c618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.601642 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.601607    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:37.734570 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.734538    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"]
Jul 26 15:29:37.738371 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:29:37.738345    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a47137a_5f31_4db0_ada4_1f9f3741c618.slice/crio-539ec97fb08727c2631df0c7bcca48e0a6c21ad94b5336c8829fd189ca9b2d24 WatchSource:0}: Error finding container 539ec97fb08727c2631df0c7bcca48e0a6c21ad94b5336c8829fd189ca9b2d24: Status 404 returned error can't find the container with id 539ec97fb08727c2631df0c7bcca48e0a6c21ad94b5336c8829fd189ca9b2d24
Jul 26 15:29:37.740038 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:37.740018    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 15:29:38.489730 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:38.489681    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms" event={"ID":"7a47137a-5f31-4db0-ada4-1f9f3741c618","Type":"ContainerStarted","Data":"89ae775211ba7194bfc23802491c05032e285daf0186046b64a7bf5b6f51592f"}
Jul 26 15:29:38.490197 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:38.489785    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms" event={"ID":"7a47137a-5f31-4db0-ada4-1f9f3741c618","Type":"ContainerStarted","Data":"539ec97fb08727c2631df0c7bcca48e0a6c21ad94b5336c8829fd189ca9b2d24"}
Jul 26 15:29:38.505547 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:38.505481    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms" podStartSLOduration=1.339875908 podStartE2EDuration="1.505463416s" totalImagesPullingTime="165.587508ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:29:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:29:38.505279479 +0000 UTC m=+662.795111598" watchObservedRunningTime="2026-07-26 15:29:38.505463416 +0000 UTC m=+662.795295534"
Jul 26 15:29:41.381096 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:41.381039    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:29:47.602674 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:47.602633    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:47.603070 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:47.602935    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:47.614749 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:47.614722    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:29:48.548034 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:29:48.548000    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qm6ms"
Jul 26 15:30:31.515180 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:30:31.515138    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:30:31.616062 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:30:31.616016    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:31:05.032174 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:05.032128    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:31:05.099924 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:05.099892    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:31:06.153314 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:06.153269    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:31:06.250356 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:06.250312    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:31:52.124107 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:52.124073    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84d47fc9db-jfnzv"]
Jul 26 15:31:52.124636 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:52.124295    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-84d47fc9db-jfnzv" podUID="940fba82-f1d4-4365-a609-ac0fe91c50d4" containerName="manager" containerID="cri-o://c3d49840c889337d82f827c877be6ee27a676d06118ab06629e38203bf0e7794" gracePeriod=30
Jul 26 15:31:52.376818 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:52.376760    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d47fc9db-jfnzv"
Jul 26 15:31:52.482398 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:52.482363    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2p8wn\" (UniqueName: \"kubernetes.io/projected/940fba82-f1d4-4365-a609-ac0fe91c50d4-kube-api-access-2p8wn\") pod \"940fba82-f1d4-4365-a609-ac0fe91c50d4\" (UID: \"940fba82-f1d4-4365-a609-ac0fe91c50d4\") "
Jul 26 15:31:52.482572 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:52.482404    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/940fba82-f1d4-4365-a609-ac0fe91c50d4-webhook-cert\") pod \"940fba82-f1d4-4365-a609-ac0fe91c50d4\" (UID: \"940fba82-f1d4-4365-a609-ac0fe91c50d4\") "
Jul 26 15:31:52.484379 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:52.484349    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/940fba82-f1d4-4365-a609-ac0fe91c50d4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "940fba82-f1d4-4365-a609-ac0fe91c50d4" (UID: "940fba82-f1d4-4365-a609-ac0fe91c50d4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:31:52.484379 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:52.484350    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/940fba82-f1d4-4365-a609-ac0fe91c50d4-kube-api-access-2p8wn" (OuterVolumeSpecName: "kube-api-access-2p8wn") pod "940fba82-f1d4-4365-a609-ac0fe91c50d4" (UID: "940fba82-f1d4-4365-a609-ac0fe91c50d4"). InnerVolumeSpecName "kube-api-access-2p8wn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:31:52.583240 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:52.583178    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2p8wn\" (UniqueName: \"kubernetes.io/projected/940fba82-f1d4-4365-a609-ac0fe91c50d4-kube-api-access-2p8wn\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:31:52.583240 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:52.583234    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/940fba82-f1d4-4365-a609-ac0fe91c50d4-webhook-cert\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:31:53.149098 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:53.149059    2559 generic.go:356] "Generic (PLEG): container finished" podID="940fba82-f1d4-4365-a609-ac0fe91c50d4" containerID="c3d49840c889337d82f827c877be6ee27a676d06118ab06629e38203bf0e7794" exitCode=0
Jul 26 15:31:53.149563 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:53.149111    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d47fc9db-jfnzv"
Jul 26 15:31:53.149563 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:53.149126    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d47fc9db-jfnzv" event={"ID":"940fba82-f1d4-4365-a609-ac0fe91c50d4","Type":"ContainerDied","Data":"c3d49840c889337d82f827c877be6ee27a676d06118ab06629e38203bf0e7794"}
Jul 26 15:31:53.149563 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:53.149178    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d47fc9db-jfnzv" event={"ID":"940fba82-f1d4-4365-a609-ac0fe91c50d4","Type":"ContainerDied","Data":"bcffa439dd8ca6209f0318d50828924c92931abd23b0fc7b25d131788591360a"}
Jul 26 15:31:53.149563 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:53.149223    2559 scope.go:117] "RemoveContainer" containerID="c3d49840c889337d82f827c877be6ee27a676d06118ab06629e38203bf0e7794"
Jul 26 15:31:53.158426 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:53.158410    2559 scope.go:117] "RemoveContainer" containerID="c3d49840c889337d82f827c877be6ee27a676d06118ab06629e38203bf0e7794"
Jul 26 15:31:53.158679 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:31:53.158662    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3d49840c889337d82f827c877be6ee27a676d06118ab06629e38203bf0e7794\": container with ID starting with c3d49840c889337d82f827c877be6ee27a676d06118ab06629e38203bf0e7794 not found: ID does not exist" containerID="c3d49840c889337d82f827c877be6ee27a676d06118ab06629e38203bf0e7794"
Jul 26 15:31:53.158723 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:53.158687    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3d49840c889337d82f827c877be6ee27a676d06118ab06629e38203bf0e7794"} err="failed to get container status \"c3d49840c889337d82f827c877be6ee27a676d06118ab06629e38203bf0e7794\": rpc error: code = NotFound desc = could not find container \"c3d49840c889337d82f827c877be6ee27a676d06118ab06629e38203bf0e7794\": container with ID starting with c3d49840c889337d82f827c877be6ee27a676d06118ab06629e38203bf0e7794 not found: ID does not exist"
Jul 26 15:31:53.169321 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:53.169295    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84d47fc9db-jfnzv"]
Jul 26 15:31:53.173173 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:53.173152    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-84d47fc9db-jfnzv"]
Jul 26 15:31:54.193154 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:31:54.193120    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="940fba82-f1d4-4365-a609-ac0fe91c50d4" path="/var/lib/kubelet/pods/940fba82-f1d4-4365-a609-ac0fe91c50d4/volumes"
Jul 26 15:32:03.157741 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:03.157621    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:32:03.213227 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:03.213182    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:32:36.190666 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:36.190630    2559 scope.go:117] "RemoveContainer" containerID="49de5177bb6ce851eb229ac2652935a4486fcd47b98bcd65c374b6d5db0cb1db"
Jul 26 15:32:53.967113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:53.967074    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84d47fc9db-d252l"]
Jul 26 15:32:53.967569 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:53.967469    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="940fba82-f1d4-4365-a609-ac0fe91c50d4" containerName="manager"
Jul 26 15:32:53.967569 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:53.967481    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="940fba82-f1d4-4365-a609-ac0fe91c50d4" containerName="manager"
Jul 26 15:32:53.967652 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:53.967601    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="940fba82-f1d4-4365-a609-ac0fe91c50d4" containerName="manager"
Jul 26 15:32:53.971019 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:53.970999    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d47fc9db-d252l"
Jul 26 15:32:53.972880 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:53.972854    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 26 15:32:53.972998 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:53.972897    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 26 15:32:53.973398 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:53.973385    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-j5ps2\""
Jul 26 15:32:53.980160 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:53.980132    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84d47fc9db-d252l"]
Jul 26 15:32:54.052476 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:54.052437    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b5dd176-2121-4b0c-8419-c0e3d5599932-webhook-cert\") pod \"maas-controller-84d47fc9db-d252l\" (UID: \"5b5dd176-2121-4b0c-8419-c0e3d5599932\") " pod="opendatahub/maas-controller-84d47fc9db-d252l"
Jul 26 15:32:54.052476 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:54.052483    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cs4q\" (UniqueName: \"kubernetes.io/projected/5b5dd176-2121-4b0c-8419-c0e3d5599932-kube-api-access-2cs4q\") pod \"maas-controller-84d47fc9db-d252l\" (UID: \"5b5dd176-2121-4b0c-8419-c0e3d5599932\") " pod="opendatahub/maas-controller-84d47fc9db-d252l"
Jul 26 15:32:54.153171 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:54.153130    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cs4q\" (UniqueName: \"kubernetes.io/projected/5b5dd176-2121-4b0c-8419-c0e3d5599932-kube-api-access-2cs4q\") pod \"maas-controller-84d47fc9db-d252l\" (UID: \"5b5dd176-2121-4b0c-8419-c0e3d5599932\") " pod="opendatahub/maas-controller-84d47fc9db-d252l"
Jul 26 15:32:54.153376 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:54.153296    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b5dd176-2121-4b0c-8419-c0e3d5599932-webhook-cert\") pod \"maas-controller-84d47fc9db-d252l\" (UID: \"5b5dd176-2121-4b0c-8419-c0e3d5599932\") " pod="opendatahub/maas-controller-84d47fc9db-d252l"
Jul 26 15:32:54.155674 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:54.155652    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b5dd176-2121-4b0c-8419-c0e3d5599932-webhook-cert\") pod \"maas-controller-84d47fc9db-d252l\" (UID: \"5b5dd176-2121-4b0c-8419-c0e3d5599932\") " pod="opendatahub/maas-controller-84d47fc9db-d252l"
Jul 26 15:32:54.160512 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:54.160480    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cs4q\" (UniqueName: \"kubernetes.io/projected/5b5dd176-2121-4b0c-8419-c0e3d5599932-kube-api-access-2cs4q\") pod \"maas-controller-84d47fc9db-d252l\" (UID: \"5b5dd176-2121-4b0c-8419-c0e3d5599932\") " pod="opendatahub/maas-controller-84d47fc9db-d252l"
Jul 26 15:32:54.282342 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:54.282255    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d47fc9db-d252l"
Jul 26 15:32:54.419972 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:54.419939    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84d47fc9db-d252l"]
Jul 26 15:32:54.439652 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:54.439619    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d47fc9db-d252l" event={"ID":"5b5dd176-2121-4b0c-8419-c0e3d5599932","Type":"ContainerStarted","Data":"fe136bf67f9536d2cdfda02affa086dd37f2edeaafc7b89410d4318c72058057"}
Jul 26 15:32:55.447173 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:55.447137    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d47fc9db-d252l" event={"ID":"5b5dd176-2121-4b0c-8419-c0e3d5599932","Type":"ContainerStarted","Data":"cf793db77450bb93e55d7d47d391cd74350e99a847f5d688de54a95cdc0e9b81"}
Jul 26 15:32:55.447596 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:55.447268    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84d47fc9db-d252l"
Jul 26 15:32:55.466492 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:32:55.466440    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-84d47fc9db-d252l" podStartSLOduration=2.069582098 podStartE2EDuration="2.466403547s" totalImagesPullingTime="396.821449ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:32:55.464459197 +0000 UTC m=+859.754291341" watchObservedRunningTime="2026-07-26 15:32:55.466403547 +0000 UTC m=+859.756235666"
Jul 26 15:33:06.455011 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:06.454977    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84d47fc9db-d252l"
Jul 26 15:33:10.953625 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:10.953591    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:33:11.064039 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:11.064005    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84d47fc9db-d252l"]
Jul 26 15:33:11.064236 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:11.064191    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-84d47fc9db-d252l" podUID="5b5dd176-2121-4b0c-8419-c0e3d5599932" containerName="manager" containerID="cri-o://cf793db77450bb93e55d7d47d391cd74350e99a847f5d688de54a95cdc0e9b81" gracePeriod=30
Jul 26 15:33:11.314178 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:11.314154    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d47fc9db-d252l"
Jul 26 15:33:11.322710 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:11.322690    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cs4q\" (UniqueName: \"kubernetes.io/projected/5b5dd176-2121-4b0c-8419-c0e3d5599932-kube-api-access-2cs4q\") pod \"5b5dd176-2121-4b0c-8419-c0e3d5599932\" (UID: \"5b5dd176-2121-4b0c-8419-c0e3d5599932\") "
Jul 26 15:33:11.322819 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:11.322746    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b5dd176-2121-4b0c-8419-c0e3d5599932-webhook-cert\") pod \"5b5dd176-2121-4b0c-8419-c0e3d5599932\" (UID: \"5b5dd176-2121-4b0c-8419-c0e3d5599932\") "
Jul 26 15:33:11.324642 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:11.324615    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b5dd176-2121-4b0c-8419-c0e3d5599932-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5b5dd176-2121-4b0c-8419-c0e3d5599932" (UID: "5b5dd176-2121-4b0c-8419-c0e3d5599932"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:33:11.324755 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:11.324670    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b5dd176-2121-4b0c-8419-c0e3d5599932-kube-api-access-2cs4q" (OuterVolumeSpecName: "kube-api-access-2cs4q") pod "5b5dd176-2121-4b0c-8419-c0e3d5599932" (UID: "5b5dd176-2121-4b0c-8419-c0e3d5599932"). InnerVolumeSpecName "kube-api-access-2cs4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:33:11.424000 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:11.423965    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cs4q\" (UniqueName: \"kubernetes.io/projected/5b5dd176-2121-4b0c-8419-c0e3d5599932-kube-api-access-2cs4q\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:33:11.424000 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:11.423997    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b5dd176-2121-4b0c-8419-c0e3d5599932-webhook-cert\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:33:11.524037 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:11.523947    2559 generic.go:356] "Generic (PLEG): container finished" podID="5b5dd176-2121-4b0c-8419-c0e3d5599932" containerID="cf793db77450bb93e55d7d47d391cd74350e99a847f5d688de54a95cdc0e9b81" exitCode=0
Jul 26 15:33:11.524037 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:11.524000    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d47fc9db-d252l"
Jul 26 15:33:11.524037 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:11.524027    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d47fc9db-d252l" event={"ID":"5b5dd176-2121-4b0c-8419-c0e3d5599932","Type":"ContainerDied","Data":"cf793db77450bb93e55d7d47d391cd74350e99a847f5d688de54a95cdc0e9b81"}
Jul 26 15:33:11.524304 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:11.524083    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d47fc9db-d252l" event={"ID":"5b5dd176-2121-4b0c-8419-c0e3d5599932","Type":"ContainerDied","Data":"fe136bf67f9536d2cdfda02affa086dd37f2edeaafc7b89410d4318c72058057"}
Jul 26 15:33:11.524304 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:11.524115    2559 scope.go:117] "RemoveContainer" containerID="cf793db77450bb93e55d7d47d391cd74350e99a847f5d688de54a95cdc0e9b81"
Jul 26 15:33:11.532980 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:11.532963    2559 scope.go:117] "RemoveContainer" containerID="cf793db77450bb93e55d7d47d391cd74350e99a847f5d688de54a95cdc0e9b81"
Jul 26 15:33:11.533235 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:33:11.533215    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf793db77450bb93e55d7d47d391cd74350e99a847f5d688de54a95cdc0e9b81\": container with ID starting with cf793db77450bb93e55d7d47d391cd74350e99a847f5d688de54a95cdc0e9b81 not found: ID does not exist" containerID="cf793db77450bb93e55d7d47d391cd74350e99a847f5d688de54a95cdc0e9b81"
Jul 26 15:33:11.533286 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:11.533243    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf793db77450bb93e55d7d47d391cd74350e99a847f5d688de54a95cdc0e9b81"} err="failed to get container status \"cf793db77450bb93e55d7d47d391cd74350e99a847f5d688de54a95cdc0e9b81\": rpc error: code = NotFound desc = could not find container \"cf793db77450bb93e55d7d47d391cd74350e99a847f5d688de54a95cdc0e9b81\": container with ID starting with cf793db77450bb93e55d7d47d391cd74350e99a847f5d688de54a95cdc0e9b81 not found: ID does not exist"
Jul 26 15:33:11.543717 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:11.543696    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84d47fc9db-d252l"]
Jul 26 15:33:11.549307 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:11.549286    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-84d47fc9db-d252l"]
Jul 26 15:33:12.194327 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.194289    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b5dd176-2121-4b0c-8419-c0e3d5599932" path="/var/lib/kubelet/pods/5b5dd176-2121-4b0c-8419-c0e3d5599932/volumes"
Jul 26 15:33:12.211329 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.211296    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:33:12.323389 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.323356    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84d47fc9db-j89xf"]
Jul 26 15:33:12.323775 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.323762    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b5dd176-2121-4b0c-8419-c0e3d5599932" containerName="manager"
Jul 26 15:33:12.323819 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.323777    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b5dd176-2121-4b0c-8419-c0e3d5599932" containerName="manager"
Jul 26 15:33:12.323865 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.323856    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b5dd176-2121-4b0c-8419-c0e3d5599932" containerName="manager"
Jul 26 15:33:12.328283 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.328258    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d47fc9db-j89xf"
Jul 26 15:33:12.329603 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.329580    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wppl9\" (UniqueName: \"kubernetes.io/projected/9b703ae5-cd78-4b6f-874c-240eee156343-kube-api-access-wppl9\") pod \"maas-controller-84d47fc9db-j89xf\" (UID: \"9b703ae5-cd78-4b6f-874c-240eee156343\") " pod="opendatahub/maas-controller-84d47fc9db-j89xf"
Jul 26 15:33:12.329723 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.329631    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b703ae5-cd78-4b6f-874c-240eee156343-webhook-cert\") pod \"maas-controller-84d47fc9db-j89xf\" (UID: \"9b703ae5-cd78-4b6f-874c-240eee156343\") " pod="opendatahub/maas-controller-84d47fc9db-j89xf"
Jul 26 15:33:12.330426 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.330409    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-j5ps2\""
Jul 26 15:33:12.331225 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.331192    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 26 15:33:12.331325 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.331228    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 26 15:33:12.336141 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.336117    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84d47fc9db-j89xf"]
Jul 26 15:33:12.430769 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.430737    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b703ae5-cd78-4b6f-874c-240eee156343-webhook-cert\") pod \"maas-controller-84d47fc9db-j89xf\" (UID: \"9b703ae5-cd78-4b6f-874c-240eee156343\") " pod="opendatahub/maas-controller-84d47fc9db-j89xf"
Jul 26 15:33:12.430943 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.430833    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wppl9\" (UniqueName: \"kubernetes.io/projected/9b703ae5-cd78-4b6f-874c-240eee156343-kube-api-access-wppl9\") pod \"maas-controller-84d47fc9db-j89xf\" (UID: \"9b703ae5-cd78-4b6f-874c-240eee156343\") " pod="opendatahub/maas-controller-84d47fc9db-j89xf"
Jul 26 15:33:12.433222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.433168    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b703ae5-cd78-4b6f-874c-240eee156343-webhook-cert\") pod \"maas-controller-84d47fc9db-j89xf\" (UID: \"9b703ae5-cd78-4b6f-874c-240eee156343\") " pod="opendatahub/maas-controller-84d47fc9db-j89xf"
Jul 26 15:33:12.437651 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.437625    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wppl9\" (UniqueName: \"kubernetes.io/projected/9b703ae5-cd78-4b6f-874c-240eee156343-kube-api-access-wppl9\") pod \"maas-controller-84d47fc9db-j89xf\" (UID: \"9b703ae5-cd78-4b6f-874c-240eee156343\") " pod="opendatahub/maas-controller-84d47fc9db-j89xf"
Jul 26 15:33:12.639763 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.639725    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d47fc9db-j89xf"
Jul 26 15:33:12.770627 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:12.770602    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84d47fc9db-j89xf"]
Jul 26 15:33:12.772893 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:33:12.772869    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b703ae5_cd78_4b6f_874c_240eee156343.slice/crio-bdc8fa9a7eef6f5c05d3aeccdaeb96975dd96ee42b24abf4b3a895294bf63db7 WatchSource:0}: Error finding container bdc8fa9a7eef6f5c05d3aeccdaeb96975dd96ee42b24abf4b3a895294bf63db7: Status 404 returned error can't find the container with id bdc8fa9a7eef6f5c05d3aeccdaeb96975dd96ee42b24abf4b3a895294bf63db7
Jul 26 15:33:13.535926 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:13.535835    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d47fc9db-j89xf" event={"ID":"9b703ae5-cd78-4b6f-874c-240eee156343","Type":"ContainerStarted","Data":"d591a5a331cf747749f78bad5474877b2af01c8795e6d38bf405cb6153a7fc78"}
Jul 26 15:33:13.535926 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:13.535920    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d47fc9db-j89xf" event={"ID":"9b703ae5-cd78-4b6f-874c-240eee156343","Type":"ContainerStarted","Data":"bdc8fa9a7eef6f5c05d3aeccdaeb96975dd96ee42b24abf4b3a895294bf63db7"}
Jul 26 15:33:13.536355 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:13.535981    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84d47fc9db-j89xf"
Jul 26 15:33:13.551771 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:13.551726    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-84d47fc9db-j89xf" podStartSLOduration=1.225999876 podStartE2EDuration="1.551710099s" totalImagesPullingTime="325.710223ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:33:13.550695193 +0000 UTC m=+877.840527337" watchObservedRunningTime="2026-07-26 15:33:13.551710099 +0000 UTC m=+877.841542218"
Jul 26 15:33:18.572923 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:18.572885    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:33:24.543340 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:24.543303    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84d47fc9db-j89xf"
Jul 26 15:33:36.215914 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:36.215878    2559 scope.go:117] "RemoveContainer" containerID="8b2168f26c0b2f075de55968cb838cff9540cb85e585f6d54b8b613590cafdda"
Jul 26 15:33:47.761407 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:47.761362    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:33:48.925003 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:48.924961    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:33:53.835499 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:53.835449    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:33:53.899343 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:53.899315    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:33:53.975755 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:53.975718    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:33:54.074630 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:54.074594    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:33:54.174693 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:54.174602    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:33:59.944608 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:33:59.944574    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:34:00.015906 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:34:00.015868    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:37:43.062353 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:37:43.062316    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:37:43.144505 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:37:43.144464    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:38:00.940393 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:38:00.940297    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:41:38.782549 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:41:38.782505    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:41:43.824190 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:41:43.824152    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:41:53.679114 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:41:53.679082    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:42:15.288409 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:42:15.288378    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:42:16.079092 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:42:16.079055    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:42:49.064536 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:42:49.064464    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:45:16.571070 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:45:16.571024    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:45:26.995136 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:45:26.995094    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:45:52.658521 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:45:52.652843    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:45:55.780169 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:45:55.780125    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:45:55.856198 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:45:55.856161    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:45:56.192508 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:45:56.192475    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:02.052429 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:02.052384    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:02.153877 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:02.153836    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:02.273695 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:02.273632    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:02.578087 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:02.578055    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:02.822506 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:02.822472    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:04.876254 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:04.876091    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:04.954284 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:04.954246    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"]
Jul 26 15:46:04.956639 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:04.956611    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:04.958428 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:04.958404    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"
Jul 26 15:46:04.960516 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:04.960496    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-6440646a-serving-cert\""
Jul 26 15:46:04.969554 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:04.969533    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"]
Jul 26 15:46:05.022932 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.022895    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:05.047681 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.047645    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c843293-4a01-4d8f-a728-f82fb8d3393d-maas-api-tls\") pod \"maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs\" (UID: \"9c843293-4a01-4d8f-a728-f82fb8d3393d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"
Jul 26 15:46:05.047853 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.047685    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c843293-4a01-4d8f-a728-f82fb8d3393d-openshift-service-ca\") pod \"maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs\" (UID: \"9c843293-4a01-4d8f-a728-f82fb8d3393d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"
Jul 26 15:46:05.047853 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.047780    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv7bj\" (UniqueName: \"kubernetes.io/projected/9c843293-4a01-4d8f-a728-f82fb8d3393d-kube-api-access-hv7bj\") pod \"maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs\" (UID: \"9c843293-4a01-4d8f-a728-f82fb8d3393d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"
Jul 26 15:46:05.148925 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.148827    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv7bj\" (UniqueName: \"kubernetes.io/projected/9c843293-4a01-4d8f-a728-f82fb8d3393d-kube-api-access-hv7bj\") pod \"maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs\" (UID: \"9c843293-4a01-4d8f-a728-f82fb8d3393d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"
Jul 26 15:46:05.149100 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.148956    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c843293-4a01-4d8f-a728-f82fb8d3393d-maas-api-tls\") pod \"maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs\" (UID: \"9c843293-4a01-4d8f-a728-f82fb8d3393d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"
Jul 26 15:46:05.149100 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.148989    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c843293-4a01-4d8f-a728-f82fb8d3393d-openshift-service-ca\") pod \"maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs\" (UID: \"9c843293-4a01-4d8f-a728-f82fb8d3393d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"
Jul 26 15:46:05.149250 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:46:05.149103    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-serving-cert: secret "maas-api-e2e-ait-6440646a-serving-cert" not found
Jul 26 15:46:05.149250 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:46:05.149179    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c843293-4a01-4d8f-a728-f82fb8d3393d-maas-api-tls podName:9c843293-4a01-4d8f-a728-f82fb8d3393d nodeName:}" failed. No retries permitted until 2026-07-26 15:46:05.649161568 +0000 UTC m=+1649.938993665 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9c843293-4a01-4d8f-a728-f82fb8d3393d-maas-api-tls") pod "maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs" (UID: "9c843293-4a01-4d8f-a728-f82fb8d3393d") : secret "maas-api-e2e-ait-6440646a-serving-cert" not found
Jul 26 15:46:05.149704 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.149681    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c843293-4a01-4d8f-a728-f82fb8d3393d-openshift-service-ca\") pod \"maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs\" (UID: \"9c843293-4a01-4d8f-a728-f82fb8d3393d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"
Jul 26 15:46:05.160527 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.160501    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv7bj\" (UniqueName: \"kubernetes.io/projected/9c843293-4a01-4d8f-a728-f82fb8d3393d-kube-api-access-hv7bj\") pod \"maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs\" (UID: \"9c843293-4a01-4d8f-a728-f82fb8d3393d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"
Jul 26 15:46:05.329510 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.329471    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:05.564678 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.564644    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:05.652396 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.652359    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c843293-4a01-4d8f-a728-f82fb8d3393d-maas-api-tls\") pod \"maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs\" (UID: \"9c843293-4a01-4d8f-a728-f82fb8d3393d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"
Jul 26 15:46:05.654775 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.654743    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c843293-4a01-4d8f-a728-f82fb8d3393d-maas-api-tls\") pod \"maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs\" (UID: \"9c843293-4a01-4d8f-a728-f82fb8d3393d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"
Jul 26 15:46:05.669478 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.669444    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:05.676397 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.676367    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2"]
Jul 26 15:46:05.680236 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.680190    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2"
Jul 26 15:46:05.682349 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.682319    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-6440646a-dockercfg-xfv5w\""
Jul 26 15:46:05.682510 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.682488    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-6440646a\""
Jul 26 15:46:05.688426 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.688401    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2"]
Jul 26 15:46:05.713839 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.713807    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz"]
Jul 26 15:46:05.717515 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.717494    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz"
Jul 26 15:46:05.728291 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.728257    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz"]
Jul 26 15:46:05.752779 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.752741    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b7c9509-05f2-42d9-9f50-11556a5986d5-plugins-config-volume\") pod \"payload-processing-e2e-ait-6440646a-7884cc9478-4stzz\" (UID: \"0b7c9509-05f2-42d9-9f50-11556a5986d5\") " pod="openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz"
Jul 26 15:46:05.752779 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.752780    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjx7m\" (UniqueName: \"kubernetes.io/projected/3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81-kube-api-access-rjx7m\") pod \"payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2\" (UID: \"3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2"
Jul 26 15:46:05.752996 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.752884    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc5g2\" (UniqueName: \"kubernetes.io/projected/0b7c9509-05f2-42d9-9f50-11556a5986d5-kube-api-access-zc5g2\") pod \"payload-processing-e2e-ait-6440646a-7884cc9478-4stzz\" (UID: \"0b7c9509-05f2-42d9-9f50-11556a5986d5\") " pod="openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz"
Jul 26 15:46:05.752996 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.752914    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2\" (UID: \"3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2"
Jul 26 15:46:05.854038 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.853938    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b7c9509-05f2-42d9-9f50-11556a5986d5-plugins-config-volume\") pod \"payload-processing-e2e-ait-6440646a-7884cc9478-4stzz\" (UID: \"0b7c9509-05f2-42d9-9f50-11556a5986d5\") " pod="openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz"
Jul 26 15:46:05.854038 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.853990    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjx7m\" (UniqueName: \"kubernetes.io/projected/3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81-kube-api-access-rjx7m\") pod \"payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2\" (UID: \"3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2"
Jul 26 15:46:05.854337 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.854052    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc5g2\" (UniqueName: \"kubernetes.io/projected/0b7c9509-05f2-42d9-9f50-11556a5986d5-kube-api-access-zc5g2\") pod \"payload-processing-e2e-ait-6440646a-7884cc9478-4stzz\" (UID: \"0b7c9509-05f2-42d9-9f50-11556a5986d5\") " pod="openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz"
Jul 26 15:46:05.854337 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.854075    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2\" (UID: \"3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2"
Jul 26 15:46:05.854827 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.854801    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2\" (UID: \"3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2"
Jul 26 15:46:05.854930 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.854907    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b7c9509-05f2-42d9-9f50-11556a5986d5-plugins-config-volume\") pod \"payload-processing-e2e-ait-6440646a-7884cc9478-4stzz\" (UID: \"0b7c9509-05f2-42d9-9f50-11556a5986d5\") " pod="openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz"
Jul 26 15:46:05.864654 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.864624    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjx7m\" (UniqueName: \"kubernetes.io/projected/3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81-kube-api-access-rjx7m\") pod \"payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2\" (UID: \"3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2"
Jul 26 15:46:05.864800 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.864634    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc5g2\" (UniqueName: \"kubernetes.io/projected/0b7c9509-05f2-42d9-9f50-11556a5986d5-kube-api-access-zc5g2\") pod \"payload-processing-e2e-ait-6440646a-7884cc9478-4stzz\" (UID: \"0b7c9509-05f2-42d9-9f50-11556a5986d5\") " pod="openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz"
Jul 26 15:46:05.870337 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.870306    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"
Jul 26 15:46:05.994144 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:05.994105    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2"
Jul 26 15:46:06.024396 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:06.024373    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"]
Jul 26 15:46:06.026743 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:46:06.026719    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c843293_4a01_4d8f_a728_f82fb8d3393d.slice/crio-0fe24ef894d0f047aca0508330acb99c54a6b5388b9f36035205c443b87119a6 WatchSource:0}: Error finding container 0fe24ef894d0f047aca0508330acb99c54a6b5388b9f36035205c443b87119a6: Status 404 returned error can't find the container with id 0fe24ef894d0f047aca0508330acb99c54a6b5388b9f36035205c443b87119a6
Jul 26 15:46:06.028512 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:06.028491    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 15:46:06.029140 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:06.029055    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz"
Jul 26 15:46:06.148876 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:06.148751    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2"]
Jul 26 15:46:06.151991 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:46:06.151958    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b488fd4_08f0_41b4_8d49_d3d6a5a5ae81.slice/crio-1d5a3cc9b4819232b2da59fa4c09447a6f710f1c839fec3bf2f3d4befcc89227 WatchSource:0}: Error finding container 1d5a3cc9b4819232b2da59fa4c09447a6f710f1c839fec3bf2f3d4befcc89227: Status 404 returned error can't find the container with id 1d5a3cc9b4819232b2da59fa4c09447a6f710f1c839fec3bf2f3d4befcc89227
Jul 26 15:46:06.180086 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:06.180065    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz"]
Jul 26 15:46:06.182824 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:46:06.182794    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b7c9509_05f2_42d9_9f50_11556a5986d5.slice/crio-614866bdfe76bbaa0d417fe6d1445b37e4080094ce2b97926c901770d3c8ffe2 WatchSource:0}: Error finding container 614866bdfe76bbaa0d417fe6d1445b37e4080094ce2b97926c901770d3c8ffe2: Status 404 returned error can't find the container with id 614866bdfe76bbaa0d417fe6d1445b37e4080094ce2b97926c901770d3c8ffe2
Jul 26 15:46:06.263017 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:06.262985    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz" event={"ID":"0b7c9509-05f2-42d9-9f50-11556a5986d5","Type":"ContainerStarted","Data":"77cfe24c660e2ae5c5005b944ebe5042fb59e4ad02c52c038bc74d23f0f4f118"}
Jul 26 15:46:06.263245 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:06.263035    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz" event={"ID":"0b7c9509-05f2-42d9-9f50-11556a5986d5","Type":"ContainerStarted","Data":"614866bdfe76bbaa0d417fe6d1445b37e4080094ce2b97926c901770d3c8ffe2"}
Jul 26 15:46:06.263245 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:06.263096    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz"
Jul 26 15:46:06.265083 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:06.264912    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2" event={"ID":"3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81","Type":"ContainerStarted","Data":"a3c34799d70c7c7ee08742bef17ebd963b7d830337c4216c8564a01595d6c43d"}
Jul 26 15:46:06.265083 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:06.264959    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2" event={"ID":"3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81","Type":"ContainerStarted","Data":"1d5a3cc9b4819232b2da59fa4c09447a6f710f1c839fec3bf2f3d4befcc89227"}
Jul 26 15:46:06.265083 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:06.264991    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2"
Jul 26 15:46:06.266196 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:06.266178    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs" event={"ID":"9c843293-4a01-4d8f-a728-f82fb8d3393d","Type":"ContainerStarted","Data":"0fe24ef894d0f047aca0508330acb99c54a6b5388b9f36035205c443b87119a6"}
Jul 26 15:46:06.279806 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:06.279752    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz" podStartSLOduration=1.27973219 podStartE2EDuration="1.27973219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:46:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:46:06.276821333 +0000 UTC m=+1650.566653445" watchObservedRunningTime="2026-07-26 15:46:06.27973219 +0000 UTC m=+1650.569564308"
Jul 26 15:46:06.292458 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:06.292407    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2" podStartSLOduration=1.292389968 podStartE2EDuration="1.292389968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:46:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:46:06.290102233 +0000 UTC m=+1650.579934350" watchObservedRunningTime="2026-07-26 15:46:06.292389968 +0000 UTC m=+1650.582222087"
Jul 26 15:46:07.273691 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:07.273657    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs" event={"ID":"9c843293-4a01-4d8f-a728-f82fb8d3393d","Type":"ContainerStarted","Data":"ec603a5df605ad02781edea43aa1792040a157b1cbdf1abc36993662af9188be"}
Jul 26 15:46:07.274148 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:07.273750    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"
Jul 26 15:46:07.289545 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:07.289494    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs" podStartSLOduration=2.842830937 podStartE2EDuration="3.289478738s" totalImagesPullingTime="446.647801ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:46:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:46:07.288473696 +0000 UTC m=+1651.578305816" watchObservedRunningTime="2026-07-26 15:46:07.289478738 +0000 UTC m=+1651.579310856"
Jul 26 15:46:09.941894 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:09.941849    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:10.043765 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:10.043733    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:13.284040 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:13.284010    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"
Jul 26 15:46:17.275623 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:17.275596    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz"
Jul 26 15:46:17.276045 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:17.275799    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2"
Jul 26 15:46:40.853439 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:40.853401    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"]
Jul 26 15:46:40.854102 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:40.853651    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs" podUID="9c843293-4a01-4d8f-a728-f82fb8d3393d" containerName="maas-api" containerID="cri-o://ec603a5df605ad02781edea43aa1792040a157b1cbdf1abc36993662af9188be" gracePeriod=30
Jul 26 15:46:40.897511 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:40.897475    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:40.935414 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:40.935379    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz"]
Jul 26 15:46:40.935700 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:40.935635    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz" podUID="0b7c9509-05f2-42d9-9f50-11556a5986d5" containerName="payload-processing" containerID="cri-o://77cfe24c660e2ae5c5005b944ebe5042fb59e4ad02c52c038bc74d23f0f4f118" gracePeriod=30
Jul 26 15:46:40.946575 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:40.946546    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2"]
Jul 26 15:46:40.946806 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:40.946778    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2" podUID="3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81" containerName="payload-pre-processing" containerID="cri-o://a3c34799d70c7c7ee08742bef17ebd963b7d830337c4216c8564a01595d6c43d" gracePeriod=30
Jul 26 15:46:41.186657 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.186632    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"
Jul 26 15:46:41.202060 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.201935    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c843293-4a01-4d8f-a728-f82fb8d3393d-openshift-service-ca\") pod \"9c843293-4a01-4d8f-a728-f82fb8d3393d\" (UID: \"9c843293-4a01-4d8f-a728-f82fb8d3393d\") "
Jul 26 15:46:41.202060 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.202010    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c843293-4a01-4d8f-a728-f82fb8d3393d-maas-api-tls\") pod \"9c843293-4a01-4d8f-a728-f82fb8d3393d\" (UID: \"9c843293-4a01-4d8f-a728-f82fb8d3393d\") "
Jul 26 15:46:41.202060 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.202035    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hv7bj\" (UniqueName: \"kubernetes.io/projected/9c843293-4a01-4d8f-a728-f82fb8d3393d-kube-api-access-hv7bj\") pod \"9c843293-4a01-4d8f-a728-f82fb8d3393d\" (UID: \"9c843293-4a01-4d8f-a728-f82fb8d3393d\") "
Jul 26 15:46:41.202666 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.202494    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c843293-4a01-4d8f-a728-f82fb8d3393d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9c843293-4a01-4d8f-a728-f82fb8d3393d" (UID: "9c843293-4a01-4d8f-a728-f82fb8d3393d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:46:41.206340 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.206280    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c843293-4a01-4d8f-a728-f82fb8d3393d-kube-api-access-hv7bj" (OuterVolumeSpecName: "kube-api-access-hv7bj") pod "9c843293-4a01-4d8f-a728-f82fb8d3393d" (UID: "9c843293-4a01-4d8f-a728-f82fb8d3393d"). InnerVolumeSpecName "kube-api-access-hv7bj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:46:41.206479 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.206410    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c843293-4a01-4d8f-a728-f82fb8d3393d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9c843293-4a01-4d8f-a728-f82fb8d3393d" (UID: "9c843293-4a01-4d8f-a728-f82fb8d3393d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:46:41.208064 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.208042    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz"
Jul 26 15:46:41.239477 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.239447    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2"
Jul 26 15:46:41.303464 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.303417    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b7c9509-05f2-42d9-9f50-11556a5986d5-plugins-config-volume\") pod \"0b7c9509-05f2-42d9-9f50-11556a5986d5\" (UID: \"0b7c9509-05f2-42d9-9f50-11556a5986d5\") "
Jul 26 15:46:41.303680 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.303482    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjx7m\" (UniqueName: \"kubernetes.io/projected/3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81-kube-api-access-rjx7m\") pod \"3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81\" (UID: \"3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81\") "
Jul 26 15:46:41.303680 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.303523    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zc5g2\" (UniqueName: \"kubernetes.io/projected/0b7c9509-05f2-42d9-9f50-11556a5986d5-kube-api-access-zc5g2\") pod \"0b7c9509-05f2-42d9-9f50-11556a5986d5\" (UID: \"0b7c9509-05f2-42d9-9f50-11556a5986d5\") "
Jul 26 15:46:41.303680 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.303599    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81-plugins-config-volume\") pod \"3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81\" (UID: \"3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81\") "
Jul 26 15:46:41.303888 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.303871    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c843293-4a01-4d8f-a728-f82fb8d3393d-maas-api-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:46:41.303949 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.303893    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hv7bj\" (UniqueName: \"kubernetes.io/projected/9c843293-4a01-4d8f-a728-f82fb8d3393d-kube-api-access-hv7bj\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:46:41.303949 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.303907    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c843293-4a01-4d8f-a728-f82fb8d3393d-openshift-service-ca\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:46:41.304071 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.304042    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b7c9509-05f2-42d9-9f50-11556a5986d5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0b7c9509-05f2-42d9-9f50-11556a5986d5" (UID: "0b7c9509-05f2-42d9-9f50-11556a5986d5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:46:41.304108 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.304042    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81" (UID: "3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:46:41.305663 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.305640    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b7c9509-05f2-42d9-9f50-11556a5986d5-kube-api-access-zc5g2" (OuterVolumeSpecName: "kube-api-access-zc5g2") pod "0b7c9509-05f2-42d9-9f50-11556a5986d5" (UID: "0b7c9509-05f2-42d9-9f50-11556a5986d5"). InnerVolumeSpecName "kube-api-access-zc5g2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:46:41.305728 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.305671    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81-kube-api-access-rjx7m" (OuterVolumeSpecName: "kube-api-access-rjx7m") pod "3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81" (UID: "3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81"). InnerVolumeSpecName "kube-api-access-rjx7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:46:41.404549 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.404463    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zc5g2\" (UniqueName: \"kubernetes.io/projected/0b7c9509-05f2-42d9-9f50-11556a5986d5-kube-api-access-zc5g2\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:46:41.404549 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.404492    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:46:41.404549 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.404502    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b7c9509-05f2-42d9-9f50-11556a5986d5-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:46:41.404549 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.404510    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjx7m\" (UniqueName: \"kubernetes.io/projected/3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81-kube-api-access-rjx7m\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:46:41.448742 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.448711    2559 generic.go:356] "Generic (PLEG): container finished" podID="0b7c9509-05f2-42d9-9f50-11556a5986d5" containerID="77cfe24c660e2ae5c5005b944ebe5042fb59e4ad02c52c038bc74d23f0f4f118" exitCode=0
Jul 26 15:46:41.448933 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.448783    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz"
Jul 26 15:46:41.448933 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.448784    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz" event={"ID":"0b7c9509-05f2-42d9-9f50-11556a5986d5","Type":"ContainerDied","Data":"77cfe24c660e2ae5c5005b944ebe5042fb59e4ad02c52c038bc74d23f0f4f118"}
Jul 26 15:46:41.449067 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.448933    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz" event={"ID":"0b7c9509-05f2-42d9-9f50-11556a5986d5","Type":"ContainerDied","Data":"614866bdfe76bbaa0d417fe6d1445b37e4080094ce2b97926c901770d3c8ffe2"}
Jul 26 15:46:41.449067 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.448966    2559 scope.go:117] "RemoveContainer" containerID="77cfe24c660e2ae5c5005b944ebe5042fb59e4ad02c52c038bc74d23f0f4f118"
Jul 26 15:46:41.450179 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.450156    2559 generic.go:356] "Generic (PLEG): container finished" podID="3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81" containerID="a3c34799d70c7c7ee08742bef17ebd963b7d830337c4216c8564a01595d6c43d" exitCode=0
Jul 26 15:46:41.450303 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.450227    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2"
Jul 26 15:46:41.450303 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.450235    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2" event={"ID":"3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81","Type":"ContainerDied","Data":"a3c34799d70c7c7ee08742bef17ebd963b7d830337c4216c8564a01595d6c43d"}
Jul 26 15:46:41.450303 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.450288    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2" event={"ID":"3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81","Type":"ContainerDied","Data":"1d5a3cc9b4819232b2da59fa4c09447a6f710f1c839fec3bf2f3d4befcc89227"}
Jul 26 15:46:41.451754 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.451733    2559 generic.go:356] "Generic (PLEG): container finished" podID="9c843293-4a01-4d8f-a728-f82fb8d3393d" containerID="ec603a5df605ad02781edea43aa1792040a157b1cbdf1abc36993662af9188be" exitCode=0
Jul 26 15:46:41.451868 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.451799    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"
Jul 26 15:46:41.451868 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.451842    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs" event={"ID":"9c843293-4a01-4d8f-a728-f82fb8d3393d","Type":"ContainerDied","Data":"ec603a5df605ad02781edea43aa1792040a157b1cbdf1abc36993662af9188be"}
Jul 26 15:46:41.451975 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.451881    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs" event={"ID":"9c843293-4a01-4d8f-a728-f82fb8d3393d","Type":"ContainerDied","Data":"0fe24ef894d0f047aca0508330acb99c54a6b5388b9f36035205c443b87119a6"}
Jul 26 15:46:41.459098 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.458988    2559 scope.go:117] "RemoveContainer" containerID="77cfe24c660e2ae5c5005b944ebe5042fb59e4ad02c52c038bc74d23f0f4f118"
Jul 26 15:46:41.459330 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:46:41.459305    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77cfe24c660e2ae5c5005b944ebe5042fb59e4ad02c52c038bc74d23f0f4f118\": container with ID starting with 77cfe24c660e2ae5c5005b944ebe5042fb59e4ad02c52c038bc74d23f0f4f118 not found: ID does not exist" containerID="77cfe24c660e2ae5c5005b944ebe5042fb59e4ad02c52c038bc74d23f0f4f118"
Jul 26 15:46:41.459413 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.459344    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77cfe24c660e2ae5c5005b944ebe5042fb59e4ad02c52c038bc74d23f0f4f118"} err="failed to get container status \"77cfe24c660e2ae5c5005b944ebe5042fb59e4ad02c52c038bc74d23f0f4f118\": rpc error: code = NotFound desc = could not find container \"77cfe24c660e2ae5c5005b944ebe5042fb59e4ad02c52c038bc74d23f0f4f118\": container with ID starting with 77cfe24c660e2ae5c5005b944ebe5042fb59e4ad02c52c038bc74d23f0f4f118 not found: ID does not exist"
Jul 26 15:46:41.459413 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.459363    2559 scope.go:117] "RemoveContainer" containerID="a3c34799d70c7c7ee08742bef17ebd963b7d830337c4216c8564a01595d6c43d"
Jul 26 15:46:41.471387 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.471330    2559 scope.go:117] "RemoveContainer" containerID="a3c34799d70c7c7ee08742bef17ebd963b7d830337c4216c8564a01595d6c43d"
Jul 26 15:46:41.471645 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:46:41.471624    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3c34799d70c7c7ee08742bef17ebd963b7d830337c4216c8564a01595d6c43d\": container with ID starting with a3c34799d70c7c7ee08742bef17ebd963b7d830337c4216c8564a01595d6c43d not found: ID does not exist" containerID="a3c34799d70c7c7ee08742bef17ebd963b7d830337c4216c8564a01595d6c43d"
Jul 26 15:46:41.471716 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.471653    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3c34799d70c7c7ee08742bef17ebd963b7d830337c4216c8564a01595d6c43d"} err="failed to get container status \"a3c34799d70c7c7ee08742bef17ebd963b7d830337c4216c8564a01595d6c43d\": rpc error: code = NotFound desc = could not find container \"a3c34799d70c7c7ee08742bef17ebd963b7d830337c4216c8564a01595d6c43d\": container with ID starting with a3c34799d70c7c7ee08742bef17ebd963b7d830337c4216c8564a01595d6c43d not found: ID does not exist"
Jul 26 15:46:41.471716 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.471671    2559 scope.go:117] "RemoveContainer" containerID="ec603a5df605ad02781edea43aa1792040a157b1cbdf1abc36993662af9188be"
Jul 26 15:46:41.472127 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.472107    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz"]
Jul 26 15:46:41.477292 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.477266    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6440646a-7884cc9478-4stzz"]
Jul 26 15:46:41.479939 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.479923    2559 scope.go:117] "RemoveContainer" containerID="ec603a5df605ad02781edea43aa1792040a157b1cbdf1abc36993662af9188be"
Jul 26 15:46:41.480237 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:46:41.480216    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec603a5df605ad02781edea43aa1792040a157b1cbdf1abc36993662af9188be\": container with ID starting with ec603a5df605ad02781edea43aa1792040a157b1cbdf1abc36993662af9188be not found: ID does not exist" containerID="ec603a5df605ad02781edea43aa1792040a157b1cbdf1abc36993662af9188be"
Jul 26 15:46:41.480307 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.480246    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec603a5df605ad02781edea43aa1792040a157b1cbdf1abc36993662af9188be"} err="failed to get container status \"ec603a5df605ad02781edea43aa1792040a157b1cbdf1abc36993662af9188be\": rpc error: code = NotFound desc = could not find container \"ec603a5df605ad02781edea43aa1792040a157b1cbdf1abc36993662af9188be\": container with ID starting with ec603a5df605ad02781edea43aa1792040a157b1cbdf1abc36993662af9188be not found: ID does not exist"
Jul 26 15:46:41.487528 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.487505    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"]
Jul 26 15:46:41.491093 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.491074    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6440646a-58f7fcbcbd-pmlcs"]
Jul 26 15:46:41.501835 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.501767    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2"]
Jul 26 15:46:41.503949 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:41.503928    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6440646a-5d9b76b656-mxln2"]
Jul 26 15:46:42.017035 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:42.017002    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:42.194393 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:42.194311    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b7c9509-05f2-42d9-9f50-11556a5986d5" path="/var/lib/kubelet/pods/0b7c9509-05f2-42d9-9f50-11556a5986d5/volumes"
Jul 26 15:46:42.194670 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:42.194658    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81" path="/var/lib/kubelet/pods/3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81/volumes"
Jul 26 15:46:42.194995 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:42.194984    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c843293-4a01-4d8f-a728-f82fb8d3393d" path="/var/lib/kubelet/pods/9c843293-4a01-4d8f-a728-f82fb8d3393d/volumes"
Jul 26 15:46:45.467963 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:45.467930    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:45.620960 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:45.620923    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:45.845240 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:45.845186    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:46.095092 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:46.095057    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:46.231814 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:46.231717    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:46.340952 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:46.340920    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:46.720958 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:46.720919    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:46.869053 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:46.869018    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:52.065332 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.065298    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:52.143570 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.143529    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"]
Jul 26 15:46:52.144134 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.144116    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81" containerName="payload-pre-processing"
Jul 26 15:46:52.144258 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.144136    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81" containerName="payload-pre-processing"
Jul 26 15:46:52.144258 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.144168    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b7c9509-05f2-42d9-9f50-11556a5986d5" containerName="payload-processing"
Jul 26 15:46:52.144258 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.144179    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b7c9509-05f2-42d9-9f50-11556a5986d5" containerName="payload-processing"
Jul 26 15:46:52.144258 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.144194    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c843293-4a01-4d8f-a728-f82fb8d3393d" containerName="maas-api"
Jul 26 15:46:52.144258 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.144225    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c843293-4a01-4d8f-a728-f82fb8d3393d" containerName="maas-api"
Jul 26 15:46:52.144528 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.144314    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c843293-4a01-4d8f-a728-f82fb8d3393d" containerName="maas-api"
Jul 26 15:46:52.144528 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.144323    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b488fd4-08f0-41b4-8d49-d3d6a5a5ae81" containerName="payload-pre-processing"
Jul 26 15:46:52.144528 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.144334    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b7c9509-05f2-42d9-9f50-11556a5986d5" containerName="payload-processing"
Jul 26 15:46:52.149570 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.149549    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"
Jul 26 15:46:52.151645 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.151626    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-3ab81599-serving-cert\""
Jul 26 15:46:52.155459 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.155440    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"]
Jul 26 15:46:52.176112 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.176078    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:52.205213 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.205165    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e30688d-d3af-46c2-924e-d2316261d0b0-maas-api-tls\") pod \"maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk\" (UID: \"9e30688d-d3af-46c2-924e-d2316261d0b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"
Jul 26 15:46:52.205213 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.205197    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e30688d-d3af-46c2-924e-d2316261d0b0-openshift-service-ca\") pod \"maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk\" (UID: \"9e30688d-d3af-46c2-924e-d2316261d0b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"
Jul 26 15:46:52.205409 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.205237    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdxgp\" (UniqueName: \"kubernetes.io/projected/9e30688d-d3af-46c2-924e-d2316261d0b0-kube-api-access-hdxgp\") pod \"maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk\" (UID: \"9e30688d-d3af-46c2-924e-d2316261d0b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"
Jul 26 15:46:52.297322 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.297283    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:52.306532 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.306506    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e30688d-d3af-46c2-924e-d2316261d0b0-maas-api-tls\") pod \"maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk\" (UID: \"9e30688d-d3af-46c2-924e-d2316261d0b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"
Jul 26 15:46:52.306662 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.306539    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e30688d-d3af-46c2-924e-d2316261d0b0-openshift-service-ca\") pod \"maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk\" (UID: \"9e30688d-d3af-46c2-924e-d2316261d0b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"
Jul 26 15:46:52.306662 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.306561    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdxgp\" (UniqueName: \"kubernetes.io/projected/9e30688d-d3af-46c2-924e-d2316261d0b0-kube-api-access-hdxgp\") pod \"maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk\" (UID: \"9e30688d-d3af-46c2-924e-d2316261d0b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"
Jul 26 15:46:52.307193 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.307168    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e30688d-d3af-46c2-924e-d2316261d0b0-openshift-service-ca\") pod \"maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk\" (UID: \"9e30688d-d3af-46c2-924e-d2316261d0b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"
Jul 26 15:46:52.308966 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.308947    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e30688d-d3af-46c2-924e-d2316261d0b0-maas-api-tls\") pod \"maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk\" (UID: \"9e30688d-d3af-46c2-924e-d2316261d0b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"
Jul 26 15:46:52.314092 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.314070    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdxgp\" (UniqueName: \"kubernetes.io/projected/9e30688d-d3af-46c2-924e-d2316261d0b0-kube-api-access-hdxgp\") pod \"maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk\" (UID: \"9e30688d-d3af-46c2-924e-d2316261d0b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"
Jul 26 15:46:52.461547 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.461449    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"
Jul 26 15:46:52.601315 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.601253    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"]
Jul 26 15:46:52.603713 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:46:52.603686    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e30688d_d3af_46c2_924e_d2316261d0b0.slice/crio-0a1c78f4b4315896a445695b6959205d45d3c264b500cfda0c31bc55a54d797e WatchSource:0}: Error finding container 0a1c78f4b4315896a445695b6959205d45d3c264b500cfda0c31bc55a54d797e: Status 404 returned error can't find the container with id 0a1c78f4b4315896a445695b6959205d45d3c264b500cfda0c31bc55a54d797e
Jul 26 15:46:52.678585 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.678544    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:52.761391 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.761259    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:46:52.768904 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.768867    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj"]
Jul 26 15:46:52.773677 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.773659    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj"
Jul 26 15:46:52.775815 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.775791    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-3ab81599\""
Jul 26 15:46:52.781498 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.781466    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj"]
Jul 26 15:46:52.810339 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.810306    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58"]
Jul 26 15:46:52.811578 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.811553    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmvqc\" (UniqueName: \"kubernetes.io/projected/50eb8432-c777-4dd7-982b-53cc3b849fe5-kube-api-access-jmvqc\") pod \"payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj\" (UID: \"50eb8432-c777-4dd7-982b-53cc3b849fe5\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj"
Jul 26 15:46:52.811702 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.811627    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50eb8432-c777-4dd7-982b-53cc3b849fe5-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj\" (UID: \"50eb8432-c777-4dd7-982b-53cc3b849fe5\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj"
Jul 26 15:46:52.814185 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.814156    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58"
Jul 26 15:46:52.816286 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.816264    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-3ab81599-dockercfg-hkdn8\""
Jul 26 15:46:52.824832 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.824806    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58"]
Jul 26 15:46:52.912856 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.912826    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spbjc\" (UniqueName: \"kubernetes.io/projected/9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9-kube-api-access-spbjc\") pod \"payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58\" (UID: \"9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9\") " pod="openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58"
Jul 26 15:46:52.912965 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.912890    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9-plugins-config-volume\") pod \"payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58\" (UID: \"9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9\") " pod="openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58"
Jul 26 15:46:52.912965 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.912951    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmvqc\" (UniqueName: \"kubernetes.io/projected/50eb8432-c777-4dd7-982b-53cc3b849fe5-kube-api-access-jmvqc\") pod \"payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj\" (UID: \"50eb8432-c777-4dd7-982b-53cc3b849fe5\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj"
Jul 26 15:46:52.913082 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.913027    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50eb8432-c777-4dd7-982b-53cc3b849fe5-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj\" (UID: \"50eb8432-c777-4dd7-982b-53cc3b849fe5\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj"
Jul 26 15:46:52.913860 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.913837    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50eb8432-c777-4dd7-982b-53cc3b849fe5-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj\" (UID: \"50eb8432-c777-4dd7-982b-53cc3b849fe5\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj"
Jul 26 15:46:52.920226 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:52.920186    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmvqc\" (UniqueName: \"kubernetes.io/projected/50eb8432-c777-4dd7-982b-53cc3b849fe5-kube-api-access-jmvqc\") pod \"payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj\" (UID: \"50eb8432-c777-4dd7-982b-53cc3b849fe5\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj"
Jul 26 15:46:53.014290 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.014185    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spbjc\" (UniqueName: \"kubernetes.io/projected/9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9-kube-api-access-spbjc\") pod \"payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58\" (UID: \"9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9\") " pod="openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58"
Jul 26 15:46:53.014290 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.014252    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9-plugins-config-volume\") pod \"payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58\" (UID: \"9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9\") " pod="openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58"
Jul 26 15:46:53.015070 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.015044    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9-plugins-config-volume\") pod \"payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58\" (UID: \"9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9\") " pod="openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58"
Jul 26 15:46:53.025197 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.025176    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spbjc\" (UniqueName: \"kubernetes.io/projected/9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9-kube-api-access-spbjc\") pod \"payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58\" (UID: \"9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9\") " pod="openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58"
Jul 26 15:46:53.085568 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.085520    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj"
Jul 26 15:46:53.126220 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.126170    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58"
Jul 26 15:46:53.239866 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.239736    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj"]
Jul 26 15:46:53.242166 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:46:53.242130    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50eb8432_c777_4dd7_982b_53cc3b849fe5.slice/crio-d74c4b5e709bd1928b885704f0c73a7e4069de985921214ae45b224329051f64 WatchSource:0}: Error finding container d74c4b5e709bd1928b885704f0c73a7e4069de985921214ae45b224329051f64: Status 404 returned error can't find the container with id d74c4b5e709bd1928b885704f0c73a7e4069de985921214ae45b224329051f64
Jul 26 15:46:53.287024 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.286999    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58"]
Jul 26 15:46:53.290426 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:46:53.290393    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f4262b8_f366_4fe2_ad5c_ad34e6f0d9f9.slice/crio-f8e7a9cc29e7cd53073454e3a76451bba0348a9673fa441ee8ad3aa62d6c6bf4 WatchSource:0}: Error finding container f8e7a9cc29e7cd53073454e3a76451bba0348a9673fa441ee8ad3aa62d6c6bf4: Status 404 returned error can't find the container with id f8e7a9cc29e7cd53073454e3a76451bba0348a9673fa441ee8ad3aa62d6c6bf4
Jul 26 15:46:53.517830 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.517796    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58" event={"ID":"9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9","Type":"ContainerStarted","Data":"d1651d6dba3a0bf6dc6d0db90f9c3966dfa46075bfdb6ed5636780b00e2b84f8"}
Jul 26 15:46:53.518027 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.517853    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58" event={"ID":"9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9","Type":"ContainerStarted","Data":"f8e7a9cc29e7cd53073454e3a76451bba0348a9673fa441ee8ad3aa62d6c6bf4"}
Jul 26 15:46:53.518027 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.517928    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58"
Jul 26 15:46:53.519437 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.519410    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk" event={"ID":"9e30688d-d3af-46c2-924e-d2316261d0b0","Type":"ContainerStarted","Data":"5fb894325c059e6021aa7a1392940471448a7917f3fbaf0514336d2506250a43"}
Jul 26 15:46:53.519570 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.519456    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk" event={"ID":"9e30688d-d3af-46c2-924e-d2316261d0b0","Type":"ContainerStarted","Data":"0a1c78f4b4315896a445695b6959205d45d3c264b500cfda0c31bc55a54d797e"}
Jul 26 15:46:53.519570 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.519487    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"
Jul 26 15:46:53.520955 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.520934    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj" event={"ID":"50eb8432-c777-4dd7-982b-53cc3b849fe5","Type":"ContainerStarted","Data":"79aeff5d9f741d11ba497445b77ab6cde1d1c46d290275146d2a1904f2ac9cb2"}
Jul 26 15:46:53.521047 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.520978    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj" event={"ID":"50eb8432-c777-4dd7-982b-53cc3b849fe5","Type":"ContainerStarted","Data":"d74c4b5e709bd1928b885704f0c73a7e4069de985921214ae45b224329051f64"}
Jul 26 15:46:53.521047 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.521011    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj"
Jul 26 15:46:53.536038 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.535942    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58" podStartSLOduration=1.5359276099999999 podStartE2EDuration="1.53592761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:46:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:46:53.534169646 +0000 UTC m=+1697.824001764" watchObservedRunningTime="2026-07-26 15:46:53.53592761 +0000 UTC m=+1697.825759769"
Jul 26 15:46:53.551446 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.551381    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk" podStartSLOduration=1.258536175 podStartE2EDuration="1.551359276s" totalImagesPullingTime="292.823101ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:46:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:46:53.54953416 +0000 UTC m=+1697.839366304" watchObservedRunningTime="2026-07-26 15:46:53.551359276 +0000 UTC m=+1697.841191395"
Jul 26 15:46:53.565915 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:53.565854    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj" podStartSLOduration=1.565832623 podStartE2EDuration="1.565832623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:46:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:46:53.563177593 +0000 UTC m=+1697.853009713" watchObservedRunningTime="2026-07-26 15:46:53.565832623 +0000 UTC m=+1697.855664745"
Jul 26 15:46:59.533317 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:46:59.533236    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"
Jul 26 15:47:04.528042 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:04.528012    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58"
Jul 26 15:47:04.528459 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:04.528313    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj"
Jul 26 15:47:27.530846 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.530809    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"]
Jul 26 15:47:27.531477 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.531024    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk" podUID="9e30688d-d3af-46c2-924e-d2316261d0b0" containerName="maas-api" containerID="cri-o://5fb894325c059e6021aa7a1392940471448a7917f3fbaf0514336d2506250a43" gracePeriod=30
Jul 26 15:47:27.561813 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.561777    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:47:27.594274 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.594234    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58"]
Jul 26 15:47:27.594723 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.594494    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58" podUID="9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9" containerName="payload-processing" containerID="cri-o://d1651d6dba3a0bf6dc6d0db90f9c3966dfa46075bfdb6ed5636780b00e2b84f8" gracePeriod=30
Jul 26 15:47:27.597784 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.597752    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj"]
Jul 26 15:47:27.598059 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.598005    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj" podUID="50eb8432-c777-4dd7-982b-53cc3b849fe5" containerName="payload-pre-processing" containerID="cri-o://79aeff5d9f741d11ba497445b77ab6cde1d1c46d290275146d2a1904f2ac9cb2" gracePeriod=30
Jul 26 15:47:27.636012 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.635978    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:47:27.694371 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.694341    2559 generic.go:356] "Generic (PLEG): container finished" podID="9e30688d-d3af-46c2-924e-d2316261d0b0" containerID="5fb894325c059e6021aa7a1392940471448a7917f3fbaf0514336d2506250a43" exitCode=0
Jul 26 15:47:27.694542 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.694412    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk" event={"ID":"9e30688d-d3af-46c2-924e-d2316261d0b0","Type":"ContainerDied","Data":"5fb894325c059e6021aa7a1392940471448a7917f3fbaf0514336d2506250a43"}
Jul 26 15:47:27.694892 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.694872    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:47:27.862511 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.862482    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"
Jul 26 15:47:27.885007 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.884981    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj"
Jul 26 15:47:27.892149 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.892122    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58"
Jul 26 15:47:27.953727 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.953692    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e30688d-d3af-46c2-924e-d2316261d0b0-openshift-service-ca\") pod \"9e30688d-d3af-46c2-924e-d2316261d0b0\" (UID: \"9e30688d-d3af-46c2-924e-d2316261d0b0\") "
Jul 26 15:47:27.953727 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.953730    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmvqc\" (UniqueName: \"kubernetes.io/projected/50eb8432-c777-4dd7-982b-53cc3b849fe5-kube-api-access-jmvqc\") pod \"50eb8432-c777-4dd7-982b-53cc3b849fe5\" (UID: \"50eb8432-c777-4dd7-982b-53cc3b849fe5\") "
Jul 26 15:47:27.953973 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.953769    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e30688d-d3af-46c2-924e-d2316261d0b0-maas-api-tls\") pod \"9e30688d-d3af-46c2-924e-d2316261d0b0\" (UID: \"9e30688d-d3af-46c2-924e-d2316261d0b0\") "
Jul 26 15:47:27.953973 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.953797    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spbjc\" (UniqueName: \"kubernetes.io/projected/9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9-kube-api-access-spbjc\") pod \"9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9\" (UID: \"9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9\") "
Jul 26 15:47:27.953973 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.953832    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9-plugins-config-volume\") pod \"9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9\" (UID: \"9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9\") "
Jul 26 15:47:27.953973 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.953904    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdxgp\" (UniqueName: \"kubernetes.io/projected/9e30688d-d3af-46c2-924e-d2316261d0b0-kube-api-access-hdxgp\") pod \"9e30688d-d3af-46c2-924e-d2316261d0b0\" (UID: \"9e30688d-d3af-46c2-924e-d2316261d0b0\") "
Jul 26 15:47:27.953973 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.953941    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50eb8432-c777-4dd7-982b-53cc3b849fe5-plugins-config-volume\") pod \"50eb8432-c777-4dd7-982b-53cc3b849fe5\" (UID: \"50eb8432-c777-4dd7-982b-53cc3b849fe5\") "
Jul 26 15:47:27.954266 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.954083    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e30688d-d3af-46c2-924e-d2316261d0b0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9e30688d-d3af-46c2-924e-d2316261d0b0" (UID: "9e30688d-d3af-46c2-924e-d2316261d0b0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:47:27.954327 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.954269    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9" (UID: "9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:47:27.954522 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.954494    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e30688d-d3af-46c2-924e-d2316261d0b0-openshift-service-ca\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:47:27.954522 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.954524    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:47:27.954750 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.954554    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50eb8432-c777-4dd7-982b-53cc3b849fe5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "50eb8432-c777-4dd7-982b-53cc3b849fe5" (UID: "50eb8432-c777-4dd7-982b-53cc3b849fe5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:47:27.955978 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.955948    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e30688d-d3af-46c2-924e-d2316261d0b0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9e30688d-d3af-46c2-924e-d2316261d0b0" (UID: "9e30688d-d3af-46c2-924e-d2316261d0b0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:47:27.956079 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.955979    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9-kube-api-access-spbjc" (OuterVolumeSpecName: "kube-api-access-spbjc") pod "9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9" (UID: "9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9"). InnerVolumeSpecName "kube-api-access-spbjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:47:27.956276 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.956248    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e30688d-d3af-46c2-924e-d2316261d0b0-kube-api-access-hdxgp" (OuterVolumeSpecName: "kube-api-access-hdxgp") pod "9e30688d-d3af-46c2-924e-d2316261d0b0" (UID: "9e30688d-d3af-46c2-924e-d2316261d0b0"). InnerVolumeSpecName "kube-api-access-hdxgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:47:27.956360 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:27.956283    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50eb8432-c777-4dd7-982b-53cc3b849fe5-kube-api-access-jmvqc" (OuterVolumeSpecName: "kube-api-access-jmvqc") pod "50eb8432-c777-4dd7-982b-53cc3b849fe5" (UID: "50eb8432-c777-4dd7-982b-53cc3b849fe5"). InnerVolumeSpecName "kube-api-access-jmvqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:47:28.055099 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.055002    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdxgp\" (UniqueName: \"kubernetes.io/projected/9e30688d-d3af-46c2-924e-d2316261d0b0-kube-api-access-hdxgp\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:47:28.055099 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.055037    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50eb8432-c777-4dd7-982b-53cc3b849fe5-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:47:28.055099 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.055048    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmvqc\" (UniqueName: \"kubernetes.io/projected/50eb8432-c777-4dd7-982b-53cc3b849fe5-kube-api-access-jmvqc\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:47:28.055099 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.055057    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e30688d-d3af-46c2-924e-d2316261d0b0-maas-api-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:47:28.055099 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.055065    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spbjc\" (UniqueName: \"kubernetes.io/projected/9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9-kube-api-access-spbjc\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:47:28.701575 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.701542    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"
Jul 26 15:47:28.701575 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.701557    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk" event={"ID":"9e30688d-d3af-46c2-924e-d2316261d0b0","Type":"ContainerDied","Data":"0a1c78f4b4315896a445695b6959205d45d3c264b500cfda0c31bc55a54d797e"}
Jul 26 15:47:28.702098 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.701621    2559 scope.go:117] "RemoveContainer" containerID="5fb894325c059e6021aa7a1392940471448a7917f3fbaf0514336d2506250a43"
Jul 26 15:47:28.702948 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.702892    2559 generic.go:356] "Generic (PLEG): container finished" podID="50eb8432-c777-4dd7-982b-53cc3b849fe5" containerID="79aeff5d9f741d11ba497445b77ab6cde1d1c46d290275146d2a1904f2ac9cb2" exitCode=0
Jul 26 15:47:28.702948 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.702926    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj"
Jul 26 15:47:28.703069 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.702981    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj" event={"ID":"50eb8432-c777-4dd7-982b-53cc3b849fe5","Type":"ContainerDied","Data":"79aeff5d9f741d11ba497445b77ab6cde1d1c46d290275146d2a1904f2ac9cb2"}
Jul 26 15:47:28.703069 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.703030    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj" event={"ID":"50eb8432-c777-4dd7-982b-53cc3b849fe5","Type":"ContainerDied","Data":"d74c4b5e709bd1928b885704f0c73a7e4069de985921214ae45b224329051f64"}
Jul 26 15:47:28.706555 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.706525    2559 generic.go:356] "Generic (PLEG): container finished" podID="9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9" containerID="d1651d6dba3a0bf6dc6d0db90f9c3966dfa46075bfdb6ed5636780b00e2b84f8" exitCode=0
Jul 26 15:47:28.706683 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.706652    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58" event={"ID":"9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9","Type":"ContainerDied","Data":"d1651d6dba3a0bf6dc6d0db90f9c3966dfa46075bfdb6ed5636780b00e2b84f8"}
Jul 26 15:47:28.706750 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.706737    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58" event={"ID":"9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9","Type":"ContainerDied","Data":"f8e7a9cc29e7cd53073454e3a76451bba0348a9673fa441ee8ad3aa62d6c6bf4"}
Jul 26 15:47:28.706846 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.706833    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58"
Jul 26 15:47:28.717663 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.717638    2559 scope.go:117] "RemoveContainer" containerID="79aeff5d9f741d11ba497445b77ab6cde1d1c46d290275146d2a1904f2ac9cb2"
Jul 26 15:47:28.726073 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.726055    2559 scope.go:117] "RemoveContainer" containerID="79aeff5d9f741d11ba497445b77ab6cde1d1c46d290275146d2a1904f2ac9cb2"
Jul 26 15:47:28.726418 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:47:28.726398    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79aeff5d9f741d11ba497445b77ab6cde1d1c46d290275146d2a1904f2ac9cb2\": container with ID starting with 79aeff5d9f741d11ba497445b77ab6cde1d1c46d290275146d2a1904f2ac9cb2 not found: ID does not exist" containerID="79aeff5d9f741d11ba497445b77ab6cde1d1c46d290275146d2a1904f2ac9cb2"
Jul 26 15:47:28.726474 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.726428    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79aeff5d9f741d11ba497445b77ab6cde1d1c46d290275146d2a1904f2ac9cb2"} err="failed to get container status \"79aeff5d9f741d11ba497445b77ab6cde1d1c46d290275146d2a1904f2ac9cb2\": rpc error: code = NotFound desc = could not find container \"79aeff5d9f741d11ba497445b77ab6cde1d1c46d290275146d2a1904f2ac9cb2\": container with ID starting with 79aeff5d9f741d11ba497445b77ab6cde1d1c46d290275146d2a1904f2ac9cb2 not found: ID does not exist"
Jul 26 15:47:28.726474 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.726447    2559 scope.go:117] "RemoveContainer" containerID="d1651d6dba3a0bf6dc6d0db90f9c3966dfa46075bfdb6ed5636780b00e2b84f8"
Jul 26 15:47:28.729697 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.729655    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58"]
Jul 26 15:47:28.735405 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.735309    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-3ab81599-5b8b47b7d9-4gl58"]
Jul 26 15:47:28.735487 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.735441    2559 scope.go:117] "RemoveContainer" containerID="d1651d6dba3a0bf6dc6d0db90f9c3966dfa46075bfdb6ed5636780b00e2b84f8"
Jul 26 15:47:28.735781 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:47:28.735754    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1651d6dba3a0bf6dc6d0db90f9c3966dfa46075bfdb6ed5636780b00e2b84f8\": container with ID starting with d1651d6dba3a0bf6dc6d0db90f9c3966dfa46075bfdb6ed5636780b00e2b84f8 not found: ID does not exist" containerID="d1651d6dba3a0bf6dc6d0db90f9c3966dfa46075bfdb6ed5636780b00e2b84f8"
Jul 26 15:47:28.735833 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.735790    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1651d6dba3a0bf6dc6d0db90f9c3966dfa46075bfdb6ed5636780b00e2b84f8"} err="failed to get container status \"d1651d6dba3a0bf6dc6d0db90f9c3966dfa46075bfdb6ed5636780b00e2b84f8\": rpc error: code = NotFound desc = could not find container \"d1651d6dba3a0bf6dc6d0db90f9c3966dfa46075bfdb6ed5636780b00e2b84f8\": container with ID starting with d1651d6dba3a0bf6dc6d0db90f9c3966dfa46075bfdb6ed5636780b00e2b84f8 not found: ID does not exist"
Jul 26 15:47:28.744621 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.744592    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"]
Jul 26 15:47:28.749602 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.749578    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3ab81599-79bdb4ccfd-r99wk"]
Jul 26 15:47:28.759343 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.759315    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj"]
Jul 26 15:47:28.764560 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:28.764533    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3ab81599-5fbb865d84-dpnmj"]
Jul 26 15:47:30.193972 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:30.193943    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50eb8432-c777-4dd7-982b-53cc3b849fe5" path="/var/lib/kubelet/pods/50eb8432-c777-4dd7-982b-53cc3b849fe5/volumes"
Jul 26 15:47:30.194373 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:30.194334    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e30688d-d3af-46c2-924e-d2316261d0b0" path="/var/lib/kubelet/pods/9e30688d-d3af-46c2-924e-d2316261d0b0/volumes"
Jul 26 15:47:30.194631 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:30.194619    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9" path="/var/lib/kubelet/pods/9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9/volumes"
Jul 26 15:47:32.042444 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:32.042400    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:47:32.195731 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:32.195692    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:47:32.429734 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:32.429701    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:47:32.653326 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:32.653289    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:47:32.977348 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:32.977309    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:47:33.473619 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:33.473579    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:47:35.072647 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.072612    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:47:35.244818 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.244782    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:47:35.331264 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.331166    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"]
Jul 26 15:47:35.331742 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.331720    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9" containerName="payload-processing"
Jul 26 15:47:35.331742 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.331741    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9" containerName="payload-processing"
Jul 26 15:47:35.331913 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.331755    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50eb8432-c777-4dd7-982b-53cc3b849fe5" containerName="payload-pre-processing"
Jul 26 15:47:35.331913 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.331764    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="50eb8432-c777-4dd7-982b-53cc3b849fe5" containerName="payload-pre-processing"
Jul 26 15:47:35.331913 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.331792    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e30688d-d3af-46c2-924e-d2316261d0b0" containerName="maas-api"
Jul 26 15:47:35.331913 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.331798    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e30688d-d3af-46c2-924e-d2316261d0b0" containerName="maas-api"
Jul 26 15:47:35.331913 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.331891    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="50eb8432-c777-4dd7-982b-53cc3b849fe5" containerName="payload-pre-processing"
Jul 26 15:47:35.331913 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.331905    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f4262b8-f366-4fe2-ad5c-ad34e6f0d9f9" containerName="payload-processing"
Jul 26 15:47:35.331913 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.331915    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e30688d-d3af-46c2-924e-d2316261d0b0" containerName="maas-api"
Jul 26 15:47:35.335046 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.335028    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"
Jul 26 15:47:35.337002 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.336979    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-35ff3b05-serving-cert\""
Jul 26 15:47:35.345196 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.345169    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"]
Jul 26 15:47:35.366117 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.366086    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:47:35.425452 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.425418    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-openshift-service-ca\") pod \"maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt\" (UID: \"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"
Jul 26 15:47:35.425612 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.425546    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-maas-api-tls\") pod \"maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt\" (UID: \"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"
Jul 26 15:47:35.425612 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.425604    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbxbx\" (UniqueName: \"kubernetes.io/projected/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-kube-api-access-vbxbx\") pod \"maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt\" (UID: \"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"
Jul 26 15:47:35.526805 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.526764    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbxbx\" (UniqueName: \"kubernetes.io/projected/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-kube-api-access-vbxbx\") pod \"maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt\" (UID: \"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"
Jul 26 15:47:35.526988 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.526830    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-openshift-service-ca\") pod \"maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt\" (UID: \"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"
Jul 26 15:47:35.526988 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.526914    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-maas-api-tls\") pod \"maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt\" (UID: \"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"
Jul 26 15:47:35.527095 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:47:35.527042    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-serving-cert: secret "maas-api-e2e-ait-35ff3b05-serving-cert" not found
Jul 26 15:47:35.527136 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:47:35.527117    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-maas-api-tls podName:a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f nodeName:}" failed. No retries permitted until 2026-07-26 15:47:36.027100186 +0000 UTC m=+1740.316932283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-maas-api-tls") pod "maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt" (UID: "a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f") : secret "maas-api-e2e-ait-35ff3b05-serving-cert" not found
Jul 26 15:47:35.527530 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.527507    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-openshift-service-ca\") pod \"maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt\" (UID: \"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"
Jul 26 15:47:35.536625 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.536593    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbxbx\" (UniqueName: \"kubernetes.io/projected/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-kube-api-access-vbxbx\") pod \"maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt\" (UID: \"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"
Jul 26 15:47:35.836649 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.836610    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:47:35.933685 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.933644    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:47:35.976408 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.976368    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c"]
Jul 26 15:47:35.980239 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.980191    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c"
Jul 26 15:47:35.982470 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.982446    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-35ff3b05\""
Jul 26 15:47:35.982596 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.982447    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-35ff3b05-dockercfg-c2mck\""
Jul 26 15:47:35.989370 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:35.989346    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c"]
Jul 26 15:47:36.008004 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.007975    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r"]
Jul 26 15:47:36.013892 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.011849    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:47:36.014171 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.014144    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r"
Jul 26 15:47:36.024105 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.024076    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r"]
Jul 26 15:47:36.032109 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.032080    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23040a06-2409-490d-965b-10432a1dcea9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c\" (UID: \"23040a06-2409-490d-965b-10432a1dcea9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c"
Jul 26 15:47:36.032244 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.032113    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv9tc\" (UniqueName: \"kubernetes.io/projected/23040a06-2409-490d-965b-10432a1dcea9-kube-api-access-qv9tc\") pod \"payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c\" (UID: \"23040a06-2409-490d-965b-10432a1dcea9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c"
Jul 26 15:47:36.032244 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.032160    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-maas-api-tls\") pod \"maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt\" (UID: \"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"
Jul 26 15:47:36.034417 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.034399    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-maas-api-tls\") pod \"maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt\" (UID: \"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"
Jul 26 15:47:36.133661 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.133572    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f0b51c5-74b8-4dcb-b976-431a2b37d323-plugins-config-volume\") pod \"payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r\" (UID: \"4f0b51c5-74b8-4dcb-b976-431a2b37d323\") " pod="openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r"
Jul 26 15:47:36.133661 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.133608    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krgq9\" (UniqueName: \"kubernetes.io/projected/4f0b51c5-74b8-4dcb-b976-431a2b37d323-kube-api-access-krgq9\") pod \"payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r\" (UID: \"4f0b51c5-74b8-4dcb-b976-431a2b37d323\") " pod="openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r"
Jul 26 15:47:36.134147 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.133703    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23040a06-2409-490d-965b-10432a1dcea9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c\" (UID: \"23040a06-2409-490d-965b-10432a1dcea9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c"
Jul 26 15:47:36.134147 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.133737    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv9tc\" (UniqueName: \"kubernetes.io/projected/23040a06-2409-490d-965b-10432a1dcea9-kube-api-access-qv9tc\") pod \"payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c\" (UID: \"23040a06-2409-490d-965b-10432a1dcea9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c"
Jul 26 15:47:36.135803 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.135785    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-35ff3b05\""
Jul 26 15:47:36.141914 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.141887    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv9tc\" (UniqueName: \"kubernetes.io/projected/23040a06-2409-490d-965b-10432a1dcea9-kube-api-access-qv9tc\") pod \"payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c\" (UID: \"23040a06-2409-490d-965b-10432a1dcea9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c"
Jul 26 15:47:36.144971 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.144947    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23040a06-2409-490d-965b-10432a1dcea9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c\" (UID: \"23040a06-2409-490d-965b-10432a1dcea9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c"
Jul 26 15:47:36.235345 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.235310    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f0b51c5-74b8-4dcb-b976-431a2b37d323-plugins-config-volume\") pod \"payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r\" (UID: \"4f0b51c5-74b8-4dcb-b976-431a2b37d323\") " pod="openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r"
Jul 26 15:47:36.235345 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.235349    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krgq9\" (UniqueName: \"kubernetes.io/projected/4f0b51c5-74b8-4dcb-b976-431a2b37d323-kube-api-access-krgq9\") pod \"payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r\" (UID: \"4f0b51c5-74b8-4dcb-b976-431a2b37d323\") " pod="openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r"
Jul 26 15:47:36.236040 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.236013    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f0b51c5-74b8-4dcb-b976-431a2b37d323-plugins-config-volume\") pod \"payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r\" (UID: \"4f0b51c5-74b8-4dcb-b976-431a2b37d323\") " pod="openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r"
Jul 26 15:47:36.243347 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.243321    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krgq9\" (UniqueName: \"kubernetes.io/projected/4f0b51c5-74b8-4dcb-b976-431a2b37d323-kube-api-access-krgq9\") pod \"payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r\" (UID: \"4f0b51c5-74b8-4dcb-b976-431a2b37d323\") " pod="openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r"
Jul 26 15:47:36.246214 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.246173    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"
Jul 26 15:47:36.295000 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.294971    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-35ff3b05-dockercfg-c2mck\""
Jul 26 15:47:36.303655 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.303629    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c"
Jul 26 15:47:36.325312 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.325273    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r"
Jul 26 15:47:36.403224 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.402964    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"]
Jul 26 15:47:36.409035 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:47:36.409006    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda82c5a2f_83c0_4dfa_9db9_15fea3c65a9f.slice/crio-30f0b4c11c75abaf408154632f095a0c7f85655d3fa48c7258a27d5b6c5d0457 WatchSource:0}: Error finding container 30f0b4c11c75abaf408154632f095a0c7f85655d3fa48c7258a27d5b6c5d0457: Status 404 returned error can't find the container with id 30f0b4c11c75abaf408154632f095a0c7f85655d3fa48c7258a27d5b6c5d0457
Jul 26 15:47:36.468689 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.468666    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c"]
Jul 26 15:47:36.472097 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:47:36.472066    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23040a06_2409_490d_965b_10432a1dcea9.slice/crio-da22b203c3625ac7385258e953cff6199172cb01a2708b50fa04fcc6522c3087 WatchSource:0}: Error finding container da22b203c3625ac7385258e953cff6199172cb01a2708b50fa04fcc6522c3087: Status 404 returned error can't find the container with id da22b203c3625ac7385258e953cff6199172cb01a2708b50fa04fcc6522c3087
Jul 26 15:47:36.694758 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.694735    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r"]
Jul 26 15:47:36.697536 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:47:36.697506    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f0b51c5_74b8_4dcb_b976_431a2b37d323.slice/crio-533b8d77a7ffda6681dab5a1f9b95b886ff4585da12e91ab89e0a4b2d74405bd WatchSource:0}: Error finding container 533b8d77a7ffda6681dab5a1f9b95b886ff4585da12e91ab89e0a4b2d74405bd: Status 404 returned error can't find the container with id 533b8d77a7ffda6681dab5a1f9b95b886ff4585da12e91ab89e0a4b2d74405bd
Jul 26 15:47:36.760764 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.760726    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r" event={"ID":"4f0b51c5-74b8-4dcb-b976-431a2b37d323","Type":"ContainerStarted","Data":"533b8d77a7ffda6681dab5a1f9b95b886ff4585da12e91ab89e0a4b2d74405bd"}
Jul 26 15:47:36.762226 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.762173    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt" event={"ID":"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f","Type":"ContainerStarted","Data":"30f0b4c11c75abaf408154632f095a0c7f85655d3fa48c7258a27d5b6c5d0457"}
Jul 26 15:47:36.764096 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.764074    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c" event={"ID":"23040a06-2409-490d-965b-10432a1dcea9","Type":"ContainerStarted","Data":"2084b8088ff58e863fc95ea127e450e20f7bfc46e858c38ccfa7370ea5d8cd9a"}
Jul 26 15:47:36.764193 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.764119    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c" event={"ID":"23040a06-2409-490d-965b-10432a1dcea9","Type":"ContainerStarted","Data":"da22b203c3625ac7385258e953cff6199172cb01a2708b50fa04fcc6522c3087"}
Jul 26 15:47:36.764274 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.764224    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c"
Jul 26 15:47:36.779513 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:36.779456    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c" podStartSLOduration=1.779441485 podStartE2EDuration="1.779441485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:47:36.778129865 +0000 UTC m=+1741.067961985" watchObservedRunningTime="2026-07-26 15:47:36.779441485 +0000 UTC m=+1741.069273602"
Jul 26 15:47:37.770679 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:37.770642    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r" event={"ID":"4f0b51c5-74b8-4dcb-b976-431a2b37d323","Type":"ContainerStarted","Data":"5ddd07b9f5f5c1736421062e577c835f78d5015865e34df7d9197fb90a064128"}
Jul 26 15:47:37.771152 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:37.770738    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r"
Jul 26 15:47:37.772398 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:37.772369    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt" event={"ID":"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f","Type":"ContainerStarted","Data":"e8a089d044362efbd2579370b8c1f990032c597775670dbd2977b3162587fd4f"}
Jul 26 15:47:37.772570 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:37.772505    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"
Jul 26 15:47:37.785728 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:37.785679    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r" podStartSLOduration=2.785662361 podStartE2EDuration="2.785662361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:47:37.784905427 +0000 UTC m=+1742.074737567" watchObservedRunningTime="2026-07-26 15:47:37.785662361 +0000 UTC m=+1742.075494481"
Jul 26 15:47:37.801135 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:37.801082    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt" podStartSLOduration=2.43245209 podStartE2EDuration="2.801063106s" totalImagesPullingTime="368.611016ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:47:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:47:37.799587742 +0000 UTC m=+1742.089419860" watchObservedRunningTime="2026-07-26 15:47:37.801063106 +0000 UTC m=+1742.090895224"
Jul 26 15:47:43.783094 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:43.783061    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"
Jul 26 15:47:47.774725 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:47.774688    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c"
Jul 26 15:47:48.778891 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:47:48.778862    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r"
Jul 26 15:48:11.275935 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.275899    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"]
Jul 26 15:48:11.276336 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.276165    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt" podUID="a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f" containerName="maas-api" containerID="cri-o://e8a089d044362efbd2579370b8c1f990032c597775670dbd2977b3162587fd4f" gracePeriod=30
Jul 26 15:48:11.304513 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.304481    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:11.334355 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.334313    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r"]
Jul 26 15:48:11.334623 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.334572    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r" podUID="4f0b51c5-74b8-4dcb-b976-431a2b37d323" containerName="payload-processing" containerID="cri-o://5ddd07b9f5f5c1736421062e577c835f78d5015865e34df7d9197fb90a064128" gracePeriod=30
Jul 26 15:48:11.341947 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.341547    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c"]
Jul 26 15:48:11.343147 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.343121    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c" podUID="23040a06-2409-490d-965b-10432a1dcea9" containerName="payload-pre-processing" containerID="cri-o://2084b8088ff58e863fc95ea127e450e20f7bfc46e858c38ccfa7370ea5d8cd9a" gracePeriod=30
Jul 26 15:48:11.375541 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.375508    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:11.443376 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.443341    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:11.615597 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.615569    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r"
Jul 26 15:48:11.619423 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.619400    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"
Jul 26 15:48:11.641232 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.641179    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c"
Jul 26 15:48:11.651788 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.651760    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-openshift-service-ca\") pod \"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f\" (UID: \"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f\") "
Jul 26 15:48:11.651964 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.651821    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krgq9\" (UniqueName: \"kubernetes.io/projected/4f0b51c5-74b8-4dcb-b976-431a2b37d323-kube-api-access-krgq9\") pod \"4f0b51c5-74b8-4dcb-b976-431a2b37d323\" (UID: \"4f0b51c5-74b8-4dcb-b976-431a2b37d323\") "
Jul 26 15:48:11.651964 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.651895    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-maas-api-tls\") pod \"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f\" (UID: \"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f\") "
Jul 26 15:48:11.651964 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.651955    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f0b51c5-74b8-4dcb-b976-431a2b37d323-plugins-config-volume\") pod \"4f0b51c5-74b8-4dcb-b976-431a2b37d323\" (UID: \"4f0b51c5-74b8-4dcb-b976-431a2b37d323\") "
Jul 26 15:48:11.652116 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.651991    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbxbx\" (UniqueName: \"kubernetes.io/projected/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-kube-api-access-vbxbx\") pod \"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f\" (UID: \"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f\") "
Jul 26 15:48:11.652362 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.652330    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f" (UID: "a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:48:11.652468 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.652442    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f0b51c5-74b8-4dcb-b976-431a2b37d323-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4f0b51c5-74b8-4dcb-b976-431a2b37d323" (UID: "4f0b51c5-74b8-4dcb-b976-431a2b37d323"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:48:11.654041 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.654011    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f" (UID: "a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:48:11.654127 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.654104    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f0b51c5-74b8-4dcb-b976-431a2b37d323-kube-api-access-krgq9" (OuterVolumeSpecName: "kube-api-access-krgq9") pod "4f0b51c5-74b8-4dcb-b976-431a2b37d323" (UID: "4f0b51c5-74b8-4dcb-b976-431a2b37d323"). InnerVolumeSpecName "kube-api-access-krgq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:48:11.654493 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.654472    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-kube-api-access-vbxbx" (OuterVolumeSpecName: "kube-api-access-vbxbx") pod "a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f" (UID: "a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f"). InnerVolumeSpecName "kube-api-access-vbxbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:48:11.752794 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.752754    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qv9tc\" (UniqueName: \"kubernetes.io/projected/23040a06-2409-490d-965b-10432a1dcea9-kube-api-access-qv9tc\") pod \"23040a06-2409-490d-965b-10432a1dcea9\" (UID: \"23040a06-2409-490d-965b-10432a1dcea9\") "
Jul 26 15:48:11.752971 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.752826    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23040a06-2409-490d-965b-10432a1dcea9-plugins-config-volume\") pod \"23040a06-2409-490d-965b-10432a1dcea9\" (UID: \"23040a06-2409-490d-965b-10432a1dcea9\") "
Jul 26 15:48:11.753052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.753001    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-maas-api-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:48:11.753052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.753012    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f0b51c5-74b8-4dcb-b976-431a2b37d323-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:48:11.753052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.753026    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbxbx\" (UniqueName: \"kubernetes.io/projected/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-kube-api-access-vbxbx\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:48:11.753052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.753038    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f-openshift-service-ca\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:48:11.753052 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.753047    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krgq9\" (UniqueName: \"kubernetes.io/projected/4f0b51c5-74b8-4dcb-b976-431a2b37d323-kube-api-access-krgq9\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:48:11.753346 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.753174    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23040a06-2409-490d-965b-10432a1dcea9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "23040a06-2409-490d-965b-10432a1dcea9" (UID: "23040a06-2409-490d-965b-10432a1dcea9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:48:11.754943 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.754920    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23040a06-2409-490d-965b-10432a1dcea9-kube-api-access-qv9tc" (OuterVolumeSpecName: "kube-api-access-qv9tc") pod "23040a06-2409-490d-965b-10432a1dcea9" (UID: "23040a06-2409-490d-965b-10432a1dcea9"). InnerVolumeSpecName "kube-api-access-qv9tc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:48:11.854358 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.854319    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23040a06-2409-490d-965b-10432a1dcea9-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:48:11.854358 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.854348    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qv9tc\" (UniqueName: \"kubernetes.io/projected/23040a06-2409-490d-965b-10432a1dcea9-kube-api-access-qv9tc\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:48:11.946360 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.946325    2559 generic.go:356] "Generic (PLEG): container finished" podID="4f0b51c5-74b8-4dcb-b976-431a2b37d323" containerID="5ddd07b9f5f5c1736421062e577c835f78d5015865e34df7d9197fb90a064128" exitCode=0
Jul 26 15:48:11.946537 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.946366    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r"
Jul 26 15:48:11.946537 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.946412    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r" event={"ID":"4f0b51c5-74b8-4dcb-b976-431a2b37d323","Type":"ContainerDied","Data":"5ddd07b9f5f5c1736421062e577c835f78d5015865e34df7d9197fb90a064128"}
Jul 26 15:48:11.946537 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.946475    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r" event={"ID":"4f0b51c5-74b8-4dcb-b976-431a2b37d323","Type":"ContainerDied","Data":"533b8d77a7ffda6681dab5a1f9b95b886ff4585da12e91ab89e0a4b2d74405bd"}
Jul 26 15:48:11.946537 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.946510    2559 scope.go:117] "RemoveContainer" containerID="5ddd07b9f5f5c1736421062e577c835f78d5015865e34df7d9197fb90a064128"
Jul 26 15:48:11.947761 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.947740    2559 generic.go:356] "Generic (PLEG): container finished" podID="a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f" containerID="e8a089d044362efbd2579370b8c1f990032c597775670dbd2977b3162587fd4f" exitCode=0
Jul 26 15:48:11.947843 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.947774    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"
Jul 26 15:48:11.947843 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.947810    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt" event={"ID":"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f","Type":"ContainerDied","Data":"e8a089d044362efbd2579370b8c1f990032c597775670dbd2977b3162587fd4f"}
Jul 26 15:48:11.947943 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.947857    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt" event={"ID":"a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f","Type":"ContainerDied","Data":"30f0b4c11c75abaf408154632f095a0c7f85655d3fa48c7258a27d5b6c5d0457"}
Jul 26 15:48:11.949190 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.949164    2559 generic.go:356] "Generic (PLEG): container finished" podID="23040a06-2409-490d-965b-10432a1dcea9" containerID="2084b8088ff58e863fc95ea127e450e20f7bfc46e858c38ccfa7370ea5d8cd9a" exitCode=0
Jul 26 15:48:11.949327 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.949196    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c"
Jul 26 15:48:11.949327 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.949247    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c" event={"ID":"23040a06-2409-490d-965b-10432a1dcea9","Type":"ContainerDied","Data":"2084b8088ff58e863fc95ea127e450e20f7bfc46e858c38ccfa7370ea5d8cd9a"}
Jul 26 15:48:11.949327 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.949282    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c" event={"ID":"23040a06-2409-490d-965b-10432a1dcea9","Type":"ContainerDied","Data":"da22b203c3625ac7385258e953cff6199172cb01a2708b50fa04fcc6522c3087"}
Jul 26 15:48:11.955906 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.955881    2559 scope.go:117] "RemoveContainer" containerID="5ddd07b9f5f5c1736421062e577c835f78d5015865e34df7d9197fb90a064128"
Jul 26 15:48:11.956188 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:48:11.956168    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ddd07b9f5f5c1736421062e577c835f78d5015865e34df7d9197fb90a064128\": container with ID starting with 5ddd07b9f5f5c1736421062e577c835f78d5015865e34df7d9197fb90a064128 not found: ID does not exist" containerID="5ddd07b9f5f5c1736421062e577c835f78d5015865e34df7d9197fb90a064128"
Jul 26 15:48:11.956292 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.956209    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ddd07b9f5f5c1736421062e577c835f78d5015865e34df7d9197fb90a064128"} err="failed to get container status \"5ddd07b9f5f5c1736421062e577c835f78d5015865e34df7d9197fb90a064128\": rpc error: code = NotFound desc = could not find container \"5ddd07b9f5f5c1736421062e577c835f78d5015865e34df7d9197fb90a064128\": container with ID starting with 5ddd07b9f5f5c1736421062e577c835f78d5015865e34df7d9197fb90a064128 not found: ID does not exist"
Jul 26 15:48:11.956292 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.956235    2559 scope.go:117] "RemoveContainer" containerID="e8a089d044362efbd2579370b8c1f990032c597775670dbd2977b3162587fd4f"
Jul 26 15:48:11.964741 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.964715    2559 scope.go:117] "RemoveContainer" containerID="e8a089d044362efbd2579370b8c1f990032c597775670dbd2977b3162587fd4f"
Jul 26 15:48:11.965009 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:48:11.964991    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8a089d044362efbd2579370b8c1f990032c597775670dbd2977b3162587fd4f\": container with ID starting with e8a089d044362efbd2579370b8c1f990032c597775670dbd2977b3162587fd4f not found: ID does not exist" containerID="e8a089d044362efbd2579370b8c1f990032c597775670dbd2977b3162587fd4f"
Jul 26 15:48:11.965096 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.965018    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8a089d044362efbd2579370b8c1f990032c597775670dbd2977b3162587fd4f"} err="failed to get container status \"e8a089d044362efbd2579370b8c1f990032c597775670dbd2977b3162587fd4f\": rpc error: code = NotFound desc = could not find container \"e8a089d044362efbd2579370b8c1f990032c597775670dbd2977b3162587fd4f\": container with ID starting with e8a089d044362efbd2579370b8c1f990032c597775670dbd2977b3162587fd4f not found: ID does not exist"
Jul 26 15:48:11.965096 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.965037    2559 scope.go:117] "RemoveContainer" containerID="2084b8088ff58e863fc95ea127e450e20f7bfc46e858c38ccfa7370ea5d8cd9a"
Jul 26 15:48:11.971603 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.971575    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r"]
Jul 26 15:48:11.973728 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.973709    2559 scope.go:117] "RemoveContainer" containerID="2084b8088ff58e863fc95ea127e450e20f7bfc46e858c38ccfa7370ea5d8cd9a"
Jul 26 15:48:11.974012 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:48:11.973992    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2084b8088ff58e863fc95ea127e450e20f7bfc46e858c38ccfa7370ea5d8cd9a\": container with ID starting with 2084b8088ff58e863fc95ea127e450e20f7bfc46e858c38ccfa7370ea5d8cd9a not found: ID does not exist" containerID="2084b8088ff58e863fc95ea127e450e20f7bfc46e858c38ccfa7370ea5d8cd9a"
Jul 26 15:48:11.974072 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.974025    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2084b8088ff58e863fc95ea127e450e20f7bfc46e858c38ccfa7370ea5d8cd9a"} err="failed to get container status \"2084b8088ff58e863fc95ea127e450e20f7bfc46e858c38ccfa7370ea5d8cd9a\": rpc error: code = NotFound desc = could not find container \"2084b8088ff58e863fc95ea127e450e20f7bfc46e858c38ccfa7370ea5d8cd9a\": container with ID starting with 2084b8088ff58e863fc95ea127e450e20f7bfc46e858c38ccfa7370ea5d8cd9a not found: ID does not exist"
Jul 26 15:48:11.974809 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.974789    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-35ff3b05-6677dbb7b8-q997r"]
Jul 26 15:48:11.983468 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.983441    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"]
Jul 26 15:48:11.986780 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.986759    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-35ff3b05-5f99b8b59-8g9gt"]
Jul 26 15:48:11.995960 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.995932    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c"]
Jul 26 15:48:11.999227 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:11.999190    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-35ff3b05-77f5d9fd68-z7l5c"]
Jul 26 15:48:12.199265 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:12.199177    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23040a06-2409-490d-965b-10432a1dcea9" path="/var/lib/kubelet/pods/23040a06-2409-490d-965b-10432a1dcea9/volumes"
Jul 26 15:48:12.199584 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:12.199569    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f0b51c5-74b8-4dcb-b976-431a2b37d323" path="/var/lib/kubelet/pods/4f0b51c5-74b8-4dcb-b976-431a2b37d323/volumes"
Jul 26 15:48:12.199906 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:12.199895    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f" path="/var/lib/kubelet/pods/a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f/volumes"
Jul 26 15:48:15.915858 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:15.915828    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:17.080864 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:17.080831    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:17.304637 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:17.304599    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:17.540411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:17.540310    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:17.824030 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:17.823995    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:18.012636 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:18.012598    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:20.320119 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.320083    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:20.413474 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.413438    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:20.515908 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.515873    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"]
Jul 26 15:48:20.516369 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.516351    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f" containerName="maas-api"
Jul 26 15:48:20.516369 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.516370    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f" containerName="maas-api"
Jul 26 15:48:20.516533 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.516385    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23040a06-2409-490d-965b-10432a1dcea9" containerName="payload-pre-processing"
Jul 26 15:48:20.516533 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.516391    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="23040a06-2409-490d-965b-10432a1dcea9" containerName="payload-pre-processing"
Jul 26 15:48:20.516533 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.516405    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f0b51c5-74b8-4dcb-b976-431a2b37d323" containerName="payload-processing"
Jul 26 15:48:20.516533 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.516410    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f0b51c5-74b8-4dcb-b976-431a2b37d323" containerName="payload-processing"
Jul 26 15:48:20.516533 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.516475    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f0b51c5-74b8-4dcb-b976-431a2b37d323" containerName="payload-processing"
Jul 26 15:48:20.516533 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.516483    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a82c5a2f-83c0-4dfa-9db9-15fea3c65a9f" containerName="maas-api"
Jul 26 15:48:20.516533 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.516491    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="23040a06-2409-490d-965b-10432a1dcea9" containerName="payload-pre-processing"
Jul 26 15:48:20.521279 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.521259    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"
Jul 26 15:48:20.523423 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.523403    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-0e86a122-serving-cert\""
Jul 26 15:48:20.531625 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.531542    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"]
Jul 26 15:48:20.551174 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.551140    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:20.634154 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.634051    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13f10f37-e39d-459e-8975-a6d28b5f0f71-maas-api-tls\") pod \"maas-api-e2e-derive-0e86a122-d6756584f-vht84\" (UID: \"13f10f37-e39d-459e-8975-a6d28b5f0f71\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"
Jul 26 15:48:20.634154 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.634093    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13f10f37-e39d-459e-8975-a6d28b5f0f71-openshift-service-ca\") pod \"maas-api-e2e-derive-0e86a122-d6756584f-vht84\" (UID: \"13f10f37-e39d-459e-8975-a6d28b5f0f71\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"
Jul 26 15:48:20.634389 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.634155    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sprbp\" (UniqueName: \"kubernetes.io/projected/13f10f37-e39d-459e-8975-a6d28b5f0f71-kube-api-access-sprbp\") pod \"maas-api-e2e-derive-0e86a122-d6756584f-vht84\" (UID: \"13f10f37-e39d-459e-8975-a6d28b5f0f71\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"
Jul 26 15:48:20.735435 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.735394    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13f10f37-e39d-459e-8975-a6d28b5f0f71-maas-api-tls\") pod \"maas-api-e2e-derive-0e86a122-d6756584f-vht84\" (UID: \"13f10f37-e39d-459e-8975-a6d28b5f0f71\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"
Jul 26 15:48:20.735435 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.735435    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13f10f37-e39d-459e-8975-a6d28b5f0f71-openshift-service-ca\") pod \"maas-api-e2e-derive-0e86a122-d6756584f-vht84\" (UID: \"13f10f37-e39d-459e-8975-a6d28b5f0f71\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"
Jul 26 15:48:20.735718 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.735470    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sprbp\" (UniqueName: \"kubernetes.io/projected/13f10f37-e39d-459e-8975-a6d28b5f0f71-kube-api-access-sprbp\") pod \"maas-api-e2e-derive-0e86a122-d6756584f-vht84\" (UID: \"13f10f37-e39d-459e-8975-a6d28b5f0f71\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"
Jul 26 15:48:20.736142 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.736119    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13f10f37-e39d-459e-8975-a6d28b5f0f71-openshift-service-ca\") pod \"maas-api-e2e-derive-0e86a122-d6756584f-vht84\" (UID: \"13f10f37-e39d-459e-8975-a6d28b5f0f71\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"
Jul 26 15:48:20.737771 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.737753    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13f10f37-e39d-459e-8975-a6d28b5f0f71-maas-api-tls\") pod \"maas-api-e2e-derive-0e86a122-d6756584f-vht84\" (UID: \"13f10f37-e39d-459e-8975-a6d28b5f0f71\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"
Jul 26 15:48:20.746975 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.746930    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:20.753053 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.753031    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sprbp\" (UniqueName: \"kubernetes.io/projected/13f10f37-e39d-459e-8975-a6d28b5f0f71-kube-api-access-sprbp\") pod \"maas-api-e2e-derive-0e86a122-d6756584f-vht84\" (UID: \"13f10f37-e39d-459e-8975-a6d28b5f0f71\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"
Jul 26 15:48:20.832777 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.832728    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"
Jul 26 15:48:20.972814 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.972787    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"]
Jul 26 15:48:20.974689 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:48:20.974659    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13f10f37_e39d_459e_8975_a6d28b5f0f71.slice/crio-65c311606479a81e0708f96cbc0238365ca71ba5abec1627dc6b2c3ce62966cb WatchSource:0}: Error finding container 65c311606479a81e0708f96cbc0238365ca71ba5abec1627dc6b2c3ce62966cb: Status 404 returned error can't find the container with id 65c311606479a81e0708f96cbc0238365ca71ba5abec1627dc6b2c3ce62966cb
Jul 26 15:48:20.997267 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:20.997225    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84" event={"ID":"13f10f37-e39d-459e-8975-a6d28b5f0f71","Type":"ContainerStarted","Data":"65c311606479a81e0708f96cbc0238365ca71ba5abec1627dc6b2c3ce62966cb"}
Jul 26 15:48:21.102863 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.102813    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:21.201711 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.201619    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:21.214554 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.214521    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm"]
Jul 26 15:48:21.219238 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.219194    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm"
Jul 26 15:48:21.221326 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.221296    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-0e86a122\""
Jul 26 15:48:21.221454 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.221355    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-0e86a122-dockercfg-kkqmv\""
Jul 26 15:48:21.225171 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.225146    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm"]
Jul 26 15:48:21.255012 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.254978    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg"]
Jul 26 15:48:21.258564 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.258548    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg"
Jul 26 15:48:21.267113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.267087    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg"]
Jul 26 15:48:21.340924 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.340896    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbeb03f9-c7fc-4abd-8b2a-083fd45def64-plugins-config-volume\") pod \"payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg\" (UID: \"dbeb03f9-c7fc-4abd-8b2a-083fd45def64\") " pod="openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg"
Jul 26 15:48:21.341312 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.340954    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz84j\" (UniqueName: \"kubernetes.io/projected/dbeb03f9-c7fc-4abd-8b2a-083fd45def64-kube-api-access-mz84j\") pod \"payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg\" (UID: \"dbeb03f9-c7fc-4abd-8b2a-083fd45def64\") " pod="openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg"
Jul 26 15:48:21.341312 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.341017    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbdaf8a8-88bb-4fd6-9a63-3360387548c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm\" (UID: \"bbdaf8a8-88bb-4fd6-9a63-3360387548c3\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm"
Jul 26 15:48:21.341312 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.341057    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdzzn\" (UniqueName: \"kubernetes.io/projected/bbdaf8a8-88bb-4fd6-9a63-3360387548c3-kube-api-access-zdzzn\") pod \"payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm\" (UID: \"bbdaf8a8-88bb-4fd6-9a63-3360387548c3\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm"
Jul 26 15:48:21.442086 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.442052    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz84j\" (UniqueName: \"kubernetes.io/projected/dbeb03f9-c7fc-4abd-8b2a-083fd45def64-kube-api-access-mz84j\") pod \"payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg\" (UID: \"dbeb03f9-c7fc-4abd-8b2a-083fd45def64\") " pod="openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg"
Jul 26 15:48:21.442086 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.442093    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbdaf8a8-88bb-4fd6-9a63-3360387548c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm\" (UID: \"bbdaf8a8-88bb-4fd6-9a63-3360387548c3\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm"
Jul 26 15:48:21.442399 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.442119    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdzzn\" (UniqueName: \"kubernetes.io/projected/bbdaf8a8-88bb-4fd6-9a63-3360387548c3-kube-api-access-zdzzn\") pod \"payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm\" (UID: \"bbdaf8a8-88bb-4fd6-9a63-3360387548c3\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm"
Jul 26 15:48:21.442399 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.442262    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbeb03f9-c7fc-4abd-8b2a-083fd45def64-plugins-config-volume\") pod \"payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg\" (UID: \"dbeb03f9-c7fc-4abd-8b2a-083fd45def64\") " pod="openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg"
Jul 26 15:48:21.442830 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.442808    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbdaf8a8-88bb-4fd6-9a63-3360387548c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm\" (UID: \"bbdaf8a8-88bb-4fd6-9a63-3360387548c3\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm"
Jul 26 15:48:21.442954 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.442935    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbeb03f9-c7fc-4abd-8b2a-083fd45def64-plugins-config-volume\") pod \"payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg\" (UID: \"dbeb03f9-c7fc-4abd-8b2a-083fd45def64\") " pod="openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg"
Jul 26 15:48:21.456838 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.456752    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz84j\" (UniqueName: \"kubernetes.io/projected/dbeb03f9-c7fc-4abd-8b2a-083fd45def64-kube-api-access-mz84j\") pod \"payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg\" (UID: \"dbeb03f9-c7fc-4abd-8b2a-083fd45def64\") " pod="openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg"
Jul 26 15:48:21.456838 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.456754    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdzzn\" (UniqueName: \"kubernetes.io/projected/bbdaf8a8-88bb-4fd6-9a63-3360387548c3-kube-api-access-zdzzn\") pod \"payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm\" (UID: \"bbdaf8a8-88bb-4fd6-9a63-3360387548c3\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm"
Jul 26 15:48:21.532166 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.532129    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm"
Jul 26 15:48:21.570086 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.570053    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg"
Jul 26 15:48:21.751608 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.751578    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg"]
Jul 26 15:48:21.753902 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:48:21.753860    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbeb03f9_c7fc_4abd_8b2a_083fd45def64.slice/crio-165a33ed93abce8e9dd46495469ee6a56bd8cfecef9f2996611136e38c4cbb60 WatchSource:0}: Error finding container 165a33ed93abce8e9dd46495469ee6a56bd8cfecef9f2996611136e38c4cbb60: Status 404 returned error can't find the container with id 165a33ed93abce8e9dd46495469ee6a56bd8cfecef9f2996611136e38c4cbb60
Jul 26 15:48:21.899583 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:21.899507    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm"]
Jul 26 15:48:21.901995 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:48:21.901966    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbdaf8a8_88bb_4fd6_9a63_3360387548c3.slice/crio-1956bc8f2e109c8a13675eadd0a764f18214a676de17ecde3f91ee92da5d6e4c WatchSource:0}: Error finding container 1956bc8f2e109c8a13675eadd0a764f18214a676de17ecde3f91ee92da5d6e4c: Status 404 returned error can't find the container with id 1956bc8f2e109c8a13675eadd0a764f18214a676de17ecde3f91ee92da5d6e4c
Jul 26 15:48:22.004639 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:22.004536    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84" event={"ID":"13f10f37-e39d-459e-8975-a6d28b5f0f71","Type":"ContainerStarted","Data":"735c5c067b2b83d126a533e7898f1ccac3571f90c8cc5fc195699145e30d247b"}
Jul 26 15:48:22.004639 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:22.004599    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"
Jul 26 15:48:22.005997 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:22.005972    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg" event={"ID":"dbeb03f9-c7fc-4abd-8b2a-083fd45def64","Type":"ContainerStarted","Data":"2bc165940f457c28928104414d2dad922639e101e2c44ee82d65ebc013be61d3"}
Jul 26 15:48:22.006103 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:22.006019    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg"
Jul 26 15:48:22.006103 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:22.006040    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg" event={"ID":"dbeb03f9-c7fc-4abd-8b2a-083fd45def64","Type":"ContainerStarted","Data":"165a33ed93abce8e9dd46495469ee6a56bd8cfecef9f2996611136e38c4cbb60"}
Jul 26 15:48:22.007382 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:22.007353    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm" event={"ID":"bbdaf8a8-88bb-4fd6-9a63-3360387548c3","Type":"ContainerStarted","Data":"7996a59f1fd09f49e443d6db86231678d1a3fd711c486b9eb476886bff907c9f"}
Jul 26 15:48:22.007483 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:22.007388    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm" event={"ID":"bbdaf8a8-88bb-4fd6-9a63-3360387548c3","Type":"ContainerStarted","Data":"1956bc8f2e109c8a13675eadd0a764f18214a676de17ecde3f91ee92da5d6e4c"}
Jul 26 15:48:22.007542 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:22.007492    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm"
Jul 26 15:48:22.023602 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:22.023558    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84" podStartSLOduration=1.6870316939999999 podStartE2EDuration="2.023543333s" totalImagesPullingTime="336.511639ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:48:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:48:22.021315585 +0000 UTC m=+1786.311147705" watchObservedRunningTime="2026-07-26 15:48:22.023543333 +0000 UTC m=+1786.313375451"
Jul 26 15:48:22.038443 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:22.038395    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm" podStartSLOduration=1.038380136 podStartE2EDuration="1.038380136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:48:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:48:22.036106514 +0000 UTC m=+1786.325938631" watchObservedRunningTime="2026-07-26 15:48:22.038380136 +0000 UTC m=+1786.328212305"
Jul 26 15:48:22.051490 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:22.051237    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg" podStartSLOduration=1.051191571 podStartE2EDuration="1.051191571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:48:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:48:22.050734882 +0000 UTC m=+1786.340567010" watchObservedRunningTime="2026-07-26 15:48:22.051191571 +0000 UTC m=+1786.341023692"
Jul 26 15:48:28.018610 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:28.018530    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"
Jul 26 15:48:33.015850 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:33.015816    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg"
Jul 26 15:48:33.016310 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:33.016092    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm"
Jul 26 15:48:41.559157 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:41.559119    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:55.707771 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:55.707721    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"]
Jul 26 15:48:55.708405 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:55.708233    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84" podUID="13f10f37-e39d-459e-8975-a6d28b5f0f71" containerName="maas-api" containerID="cri-o://735c5c067b2b83d126a533e7898f1ccac3571f90c8cc5fc195699145e30d247b" gracePeriod=30
Jul 26 15:48:55.734579 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:55.734544    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:55.758323 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:55.758284    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg"]
Jul 26 15:48:55.758539 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:55.758494    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg" podUID="dbeb03f9-c7fc-4abd-8b2a-083fd45def64" containerName="payload-processing" containerID="cri-o://2bc165940f457c28928104414d2dad922639e101e2c44ee82d65ebc013be61d3" gracePeriod=30
Jul 26 15:48:55.769308 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:55.768688    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm"]
Jul 26 15:48:55.769308 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:55.768923    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm" podUID="bbdaf8a8-88bb-4fd6-9a63-3360387548c3" containerName="payload-pre-processing" containerID="cri-o://7996a59f1fd09f49e443d6db86231678d1a3fd711c486b9eb476886bff907c9f" gracePeriod=30
Jul 26 15:48:55.834956 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:55.834919    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:55.886957 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:55.886925    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:48:56.051415 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.051384    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg"
Jul 26 15:48:56.053518 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.053499    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"
Jul 26 15:48:56.059649 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.059626    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm"
Jul 26 15:48:56.175176 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.175135    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mz84j\" (UniqueName: \"kubernetes.io/projected/dbeb03f9-c7fc-4abd-8b2a-083fd45def64-kube-api-access-mz84j\") pod \"dbeb03f9-c7fc-4abd-8b2a-083fd45def64\" (UID: \"dbeb03f9-c7fc-4abd-8b2a-083fd45def64\") "
Jul 26 15:48:56.175176 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.175181    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdzzn\" (UniqueName: \"kubernetes.io/projected/bbdaf8a8-88bb-4fd6-9a63-3360387548c3-kube-api-access-zdzzn\") pod \"bbdaf8a8-88bb-4fd6-9a63-3360387548c3\" (UID: \"bbdaf8a8-88bb-4fd6-9a63-3360387548c3\") "
Jul 26 15:48:56.175453 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.175229    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sprbp\" (UniqueName: \"kubernetes.io/projected/13f10f37-e39d-459e-8975-a6d28b5f0f71-kube-api-access-sprbp\") pod \"13f10f37-e39d-459e-8975-a6d28b5f0f71\" (UID: \"13f10f37-e39d-459e-8975-a6d28b5f0f71\") "
Jul 26 15:48:56.175453 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.175269    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13f10f37-e39d-459e-8975-a6d28b5f0f71-openshift-service-ca\") pod \"13f10f37-e39d-459e-8975-a6d28b5f0f71\" (UID: \"13f10f37-e39d-459e-8975-a6d28b5f0f71\") "
Jul 26 15:48:56.175453 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.175315    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13f10f37-e39d-459e-8975-a6d28b5f0f71-maas-api-tls\") pod \"13f10f37-e39d-459e-8975-a6d28b5f0f71\" (UID: \"13f10f37-e39d-459e-8975-a6d28b5f0f71\") "
Jul 26 15:48:56.175453 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.175338    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbeb03f9-c7fc-4abd-8b2a-083fd45def64-plugins-config-volume\") pod \"dbeb03f9-c7fc-4abd-8b2a-083fd45def64\" (UID: \"dbeb03f9-c7fc-4abd-8b2a-083fd45def64\") "
Jul 26 15:48:56.175453 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.175372    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbdaf8a8-88bb-4fd6-9a63-3360387548c3-plugins-config-volume\") pod \"bbdaf8a8-88bb-4fd6-9a63-3360387548c3\" (UID: \"bbdaf8a8-88bb-4fd6-9a63-3360387548c3\") "
Jul 26 15:48:56.175795 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.175759    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbeb03f9-c7fc-4abd-8b2a-083fd45def64-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dbeb03f9-c7fc-4abd-8b2a-083fd45def64" (UID: "dbeb03f9-c7fc-4abd-8b2a-083fd45def64"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:48:56.175920 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.175764    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13f10f37-e39d-459e-8975-a6d28b5f0f71-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "13f10f37-e39d-459e-8975-a6d28b5f0f71" (UID: "13f10f37-e39d-459e-8975-a6d28b5f0f71"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:48:56.176046 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.176021    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbdaf8a8-88bb-4fd6-9a63-3360387548c3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bbdaf8a8-88bb-4fd6-9a63-3360387548c3" (UID: "bbdaf8a8-88bb-4fd6-9a63-3360387548c3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:48:56.177496 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.177476    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbdaf8a8-88bb-4fd6-9a63-3360387548c3-kube-api-access-zdzzn" (OuterVolumeSpecName: "kube-api-access-zdzzn") pod "bbdaf8a8-88bb-4fd6-9a63-3360387548c3" (UID: "bbdaf8a8-88bb-4fd6-9a63-3360387548c3"). InnerVolumeSpecName "kube-api-access-zdzzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:48:56.177808 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.177789    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13f10f37-e39d-459e-8975-a6d28b5f0f71-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "13f10f37-e39d-459e-8975-a6d28b5f0f71" (UID: "13f10f37-e39d-459e-8975-a6d28b5f0f71"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:48:56.177874 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.177845    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13f10f37-e39d-459e-8975-a6d28b5f0f71-kube-api-access-sprbp" (OuterVolumeSpecName: "kube-api-access-sprbp") pod "13f10f37-e39d-459e-8975-a6d28b5f0f71" (UID: "13f10f37-e39d-459e-8975-a6d28b5f0f71"). InnerVolumeSpecName "kube-api-access-sprbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:48:56.177945 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.177927    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbeb03f9-c7fc-4abd-8b2a-083fd45def64-kube-api-access-mz84j" (OuterVolumeSpecName: "kube-api-access-mz84j") pod "dbeb03f9-c7fc-4abd-8b2a-083fd45def64" (UID: "dbeb03f9-c7fc-4abd-8b2a-083fd45def64"). InnerVolumeSpecName "kube-api-access-mz84j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:48:56.182595 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.182571    2559 generic.go:356] "Generic (PLEG): container finished" podID="13f10f37-e39d-459e-8975-a6d28b5f0f71" containerID="735c5c067b2b83d126a533e7898f1ccac3571f90c8cc5fc195699145e30d247b" exitCode=0
Jul 26 15:48:56.182683 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.182619    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"
Jul 26 15:48:56.182683 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.182646    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84" event={"ID":"13f10f37-e39d-459e-8975-a6d28b5f0f71","Type":"ContainerDied","Data":"735c5c067b2b83d126a533e7898f1ccac3571f90c8cc5fc195699145e30d247b"}
Jul 26 15:48:56.182774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.182698    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84" event={"ID":"13f10f37-e39d-459e-8975-a6d28b5f0f71","Type":"ContainerDied","Data":"65c311606479a81e0708f96cbc0238365ca71ba5abec1627dc6b2c3ce62966cb"}
Jul 26 15:48:56.182774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.182720    2559 scope.go:117] "RemoveContainer" containerID="735c5c067b2b83d126a533e7898f1ccac3571f90c8cc5fc195699145e30d247b"
Jul 26 15:48:56.183952 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.183922    2559 generic.go:356] "Generic (PLEG): container finished" podID="dbeb03f9-c7fc-4abd-8b2a-083fd45def64" containerID="2bc165940f457c28928104414d2dad922639e101e2c44ee82d65ebc013be61d3" exitCode=0
Jul 26 15:48:56.184046 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.183959    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg"
Jul 26 15:48:56.184046 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.184004    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg" event={"ID":"dbeb03f9-c7fc-4abd-8b2a-083fd45def64","Type":"ContainerDied","Data":"2bc165940f457c28928104414d2dad922639e101e2c44ee82d65ebc013be61d3"}
Jul 26 15:48:56.184166 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.184053    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg" event={"ID":"dbeb03f9-c7fc-4abd-8b2a-083fd45def64","Type":"ContainerDied","Data":"165a33ed93abce8e9dd46495469ee6a56bd8cfecef9f2996611136e38c4cbb60"}
Jul 26 15:48:56.185488 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.185453    2559 generic.go:356] "Generic (PLEG): container finished" podID="bbdaf8a8-88bb-4fd6-9a63-3360387548c3" containerID="7996a59f1fd09f49e443d6db86231678d1a3fd711c486b9eb476886bff907c9f" exitCode=0
Jul 26 15:48:56.185587 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.185496    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm"
Jul 26 15:48:56.185587 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.185505    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm" event={"ID":"bbdaf8a8-88bb-4fd6-9a63-3360387548c3","Type":"ContainerDied","Data":"7996a59f1fd09f49e443d6db86231678d1a3fd711c486b9eb476886bff907c9f"}
Jul 26 15:48:56.185587 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.185544    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm" event={"ID":"bbdaf8a8-88bb-4fd6-9a63-3360387548c3","Type":"ContainerDied","Data":"1956bc8f2e109c8a13675eadd0a764f18214a676de17ecde3f91ee92da5d6e4c"}
Jul 26 15:48:56.193423 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.193334    2559 scope.go:117] "RemoveContainer" containerID="735c5c067b2b83d126a533e7898f1ccac3571f90c8cc5fc195699145e30d247b"
Jul 26 15:48:56.193625 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:48:56.193603    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"735c5c067b2b83d126a533e7898f1ccac3571f90c8cc5fc195699145e30d247b\": container with ID starting with 735c5c067b2b83d126a533e7898f1ccac3571f90c8cc5fc195699145e30d247b not found: ID does not exist" containerID="735c5c067b2b83d126a533e7898f1ccac3571f90c8cc5fc195699145e30d247b"
Jul 26 15:48:56.193700 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.193638    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"735c5c067b2b83d126a533e7898f1ccac3571f90c8cc5fc195699145e30d247b"} err="failed to get container status \"735c5c067b2b83d126a533e7898f1ccac3571f90c8cc5fc195699145e30d247b\": rpc error: code = NotFound desc = could not find container \"735c5c067b2b83d126a533e7898f1ccac3571f90c8cc5fc195699145e30d247b\": container with ID starting with 735c5c067b2b83d126a533e7898f1ccac3571f90c8cc5fc195699145e30d247b not found: ID does not exist"
Jul 26 15:48:56.193700 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.193662    2559 scope.go:117] "RemoveContainer" containerID="2bc165940f457c28928104414d2dad922639e101e2c44ee82d65ebc013be61d3"
Jul 26 15:48:56.203577 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.203560    2559 scope.go:117] "RemoveContainer" containerID="2bc165940f457c28928104414d2dad922639e101e2c44ee82d65ebc013be61d3"
Jul 26 15:48:56.203836 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:48:56.203820    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bc165940f457c28928104414d2dad922639e101e2c44ee82d65ebc013be61d3\": container with ID starting with 2bc165940f457c28928104414d2dad922639e101e2c44ee82d65ebc013be61d3 not found: ID does not exist" containerID="2bc165940f457c28928104414d2dad922639e101e2c44ee82d65ebc013be61d3"
Jul 26 15:48:56.203877 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.203842    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bc165940f457c28928104414d2dad922639e101e2c44ee82d65ebc013be61d3"} err="failed to get container status \"2bc165940f457c28928104414d2dad922639e101e2c44ee82d65ebc013be61d3\": rpc error: code = NotFound desc = could not find container \"2bc165940f457c28928104414d2dad922639e101e2c44ee82d65ebc013be61d3\": container with ID starting with 2bc165940f457c28928104414d2dad922639e101e2c44ee82d65ebc013be61d3 not found: ID does not exist"
Jul 26 15:48:56.203877 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.203860    2559 scope.go:117] "RemoveContainer" containerID="7996a59f1fd09f49e443d6db86231678d1a3fd711c486b9eb476886bff907c9f"
Jul 26 15:48:56.211291 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.209531    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg"]
Jul 26 15:48:56.212868 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.212849    2559 scope.go:117] "RemoveContainer" containerID="7996a59f1fd09f49e443d6db86231678d1a3fd711c486b9eb476886bff907c9f"
Jul 26 15:48:56.213379 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:48:56.213274    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7996a59f1fd09f49e443d6db86231678d1a3fd711c486b9eb476886bff907c9f\": container with ID starting with 7996a59f1fd09f49e443d6db86231678d1a3fd711c486b9eb476886bff907c9f not found: ID does not exist" containerID="7996a59f1fd09f49e443d6db86231678d1a3fd711c486b9eb476886bff907c9f"
Jul 26 15:48:56.213379 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.213337    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7996a59f1fd09f49e443d6db86231678d1a3fd711c486b9eb476886bff907c9f"} err="failed to get container status \"7996a59f1fd09f49e443d6db86231678d1a3fd711c486b9eb476886bff907c9f\": rpc error: code = NotFound desc = could not find container \"7996a59f1fd09f49e443d6db86231678d1a3fd711c486b9eb476886bff907c9f\": container with ID starting with 7996a59f1fd09f49e443d6db86231678d1a3fd711c486b9eb476886bff907c9f not found: ID does not exist"
Jul 26 15:48:56.215933 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.215904    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0e86a122-7c4999df46-cp9qg"]
Jul 26 15:48:56.228048 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.227958    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"]
Jul 26 15:48:56.230084 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.230059    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0e86a122-d6756584f-vht84"]
Jul 26 15:48:56.238935 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.238915    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm"]
Jul 26 15:48:56.242188 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.242169    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0e86a122-69bcc7b9b-z9zsm"]
Jul 26 15:48:56.276614 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.276583    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13f10f37-e39d-459e-8975-a6d28b5f0f71-openshift-service-ca\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:48:56.276614 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.276610    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13f10f37-e39d-459e-8975-a6d28b5f0f71-maas-api-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:48:56.276765 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.276622    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbeb03f9-c7fc-4abd-8b2a-083fd45def64-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:48:56.276765 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.276630    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbdaf8a8-88bb-4fd6-9a63-3360387548c3-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:48:56.276765 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.276639    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mz84j\" (UniqueName: \"kubernetes.io/projected/dbeb03f9-c7fc-4abd-8b2a-083fd45def64-kube-api-access-mz84j\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:48:56.276765 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.276647    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdzzn\" (UniqueName: \"kubernetes.io/projected/bbdaf8a8-88bb-4fd6-9a63-3360387548c3-kube-api-access-zdzzn\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:48:56.276765 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:56.276654    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sprbp\" (UniqueName: \"kubernetes.io/projected/13f10f37-e39d-459e-8975-a6d28b5f0f71-kube-api-access-sprbp\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:48:58.194123 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:58.194087    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13f10f37-e39d-459e-8975-a6d28b5f0f71" path="/var/lib/kubelet/pods/13f10f37-e39d-459e-8975-a6d28b5f0f71/volumes"
Jul 26 15:48:58.194565 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:58.194505    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbdaf8a8-88bb-4fd6-9a63-3360387548c3" path="/var/lib/kubelet/pods/bbdaf8a8-88bb-4fd6-9a63-3360387548c3/volumes"
Jul 26 15:48:58.194813 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:48:58.194799    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbeb03f9-c7fc-4abd-8b2a-083fd45def64" path="/var/lib/kubelet/pods/dbeb03f9-c7fc-4abd-8b2a-083fd45def64/volumes"
Jul 26 15:49:00.252002 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:49:00.251970    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:49:00.404264 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:49:00.404185    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:49:00.641319 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:49:00.641278    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:49:01.121599 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:49:01.121563    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:49:01.217147 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:49:01.217108    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:49:01.317503 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:49:01.317471    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:49:02.687617 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:49:02.687572    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:49:08.365337 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:49:08.365299    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:49:08.468494 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:49:08.468464    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:49:08.571614 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:49:08.571576    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:49:08.667978 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:49:08.667887    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:49:09.060175 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:49:09.060099    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:49:25.671711 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:49:25.671668    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:49:35.019733 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:49:35.019690    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:49:41.570581 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:49:41.570549    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:49:41.679386 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:49:41.679344    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:49:41.772893 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:49:41.772860    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:49:57.634402 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:49:57.634355    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:03.426608 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:03.426571    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:03.496284 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:03.496247    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:03.591462 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:03.591425    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:03.692267 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:03.692147    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:14.157273 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:14.157229    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:14.910758 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:14.910714    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:20.644629 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:20.644593    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:20.753676 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:20.753642    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:20.843591 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:20.843526    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:20.942700 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:20.942619    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:21.809005 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:21.808968    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:21.887853 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:21.887820    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:23.684065 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:23.684025    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:29.408125 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:29.408089    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:29.466028 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:29.465994    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:30.072489 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:30.072446    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:36.079089 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:36.079047    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:36.176635 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:36.176595    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:36.325978 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:36.325919    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:36.651354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:36.651319    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:36.954606 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:36.954490    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:39.539815 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.539774    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:39.624395 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.624360    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:39.625865 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.625835    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"]
Jul 26 15:50:39.626514 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.626487    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13f10f37-e39d-459e-8975-a6d28b5f0f71" containerName="maas-api"
Jul 26 15:50:39.626514 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.626516    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="13f10f37-e39d-459e-8975-a6d28b5f0f71" containerName="maas-api"
Jul 26 15:50:39.626662 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.626529    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbeb03f9-c7fc-4abd-8b2a-083fd45def64" containerName="payload-processing"
Jul 26 15:50:39.626662 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.626539    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbeb03f9-c7fc-4abd-8b2a-083fd45def64" containerName="payload-processing"
Jul 26 15:50:39.626662 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.626574    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbdaf8a8-88bb-4fd6-9a63-3360387548c3" containerName="payload-pre-processing"
Jul 26 15:50:39.626662 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.626582    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbdaf8a8-88bb-4fd6-9a63-3360387548c3" containerName="payload-pre-processing"
Jul 26 15:50:39.626833 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.626675    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="13f10f37-e39d-459e-8975-a6d28b5f0f71" containerName="maas-api"
Jul 26 15:50:39.626833 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.626688    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbeb03f9-c7fc-4abd-8b2a-083fd45def64" containerName="payload-processing"
Jul 26 15:50:39.626833 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.626697    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbdaf8a8-88bb-4fd6-9a63-3360387548c3" containerName="payload-pre-processing"
Jul 26 15:50:39.630241 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.630219    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"
Jul 26 15:50:39.632101 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.632077    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4debde71-serving-cert\""
Jul 26 15:50:39.637568 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.637541    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"]
Jul 26 15:50:39.698488 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.698452    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:39.755196 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.755154    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4059dd2e-1553-47fd-9ee0-e073eac435a3-maas-api-tls\") pod \"maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs\" (UID: \"4059dd2e-1553-47fd-9ee0-e073eac435a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"
Jul 26 15:50:39.755416 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.755228    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhll2\" (UniqueName: \"kubernetes.io/projected/4059dd2e-1553-47fd-9ee0-e073eac435a3-kube-api-access-vhll2\") pod \"maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs\" (UID: \"4059dd2e-1553-47fd-9ee0-e073eac435a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"
Jul 26 15:50:39.755416 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.755271    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4059dd2e-1553-47fd-9ee0-e073eac435a3-openshift-service-ca\") pod \"maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs\" (UID: \"4059dd2e-1553-47fd-9ee0-e073eac435a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"
Jul 26 15:50:39.856152 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.856109    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4059dd2e-1553-47fd-9ee0-e073eac435a3-maas-api-tls\") pod \"maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs\" (UID: \"4059dd2e-1553-47fd-9ee0-e073eac435a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"
Jul 26 15:50:39.856152 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.856160    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhll2\" (UniqueName: \"kubernetes.io/projected/4059dd2e-1553-47fd-9ee0-e073eac435a3-kube-api-access-vhll2\") pod \"maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs\" (UID: \"4059dd2e-1553-47fd-9ee0-e073eac435a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"
Jul 26 15:50:39.856399 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.856187    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4059dd2e-1553-47fd-9ee0-e073eac435a3-openshift-service-ca\") pod \"maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs\" (UID: \"4059dd2e-1553-47fd-9ee0-e073eac435a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"
Jul 26 15:50:39.856399 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:50:39.856288    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-serving-cert: secret "maas-api-e2e-mt-4debde71-serving-cert" not found
Jul 26 15:50:39.856399 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:50:39.856365    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4059dd2e-1553-47fd-9ee0-e073eac435a3-maas-api-tls podName:4059dd2e-1553-47fd-9ee0-e073eac435a3 nodeName:}" failed. No retries permitted until 2026-07-26 15:50:40.356344567 +0000 UTC m=+1924.646176665 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4059dd2e-1553-47fd-9ee0-e073eac435a3-maas-api-tls") pod "maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs" (UID: "4059dd2e-1553-47fd-9ee0-e073eac435a3") : secret "maas-api-e2e-mt-4debde71-serving-cert" not found
Jul 26 15:50:39.856798 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.856780    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4059dd2e-1553-47fd-9ee0-e073eac435a3-openshift-service-ca\") pod \"maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs\" (UID: \"4059dd2e-1553-47fd-9ee0-e073eac435a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"
Jul 26 15:50:39.866855 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:39.866830    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhll2\" (UniqueName: \"kubernetes.io/projected/4059dd2e-1553-47fd-9ee0-e073eac435a3-kube-api-access-vhll2\") pod \"maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs\" (UID: \"4059dd2e-1553-47fd-9ee0-e073eac435a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"
Jul 26 15:50:40.247946 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.247848    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:40.325765 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.325730    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:40.361427 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.361393    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4059dd2e-1553-47fd-9ee0-e073eac435a3-maas-api-tls\") pod \"maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs\" (UID: \"4059dd2e-1553-47fd-9ee0-e073eac435a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"
Jul 26 15:50:40.361673 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:50:40.361559    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-serving-cert: secret "maas-api-e2e-mt-4debde71-serving-cert" not found
Jul 26 15:50:40.361673 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:50:40.361639    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4059dd2e-1553-47fd-9ee0-e073eac435a3-maas-api-tls podName:4059dd2e-1553-47fd-9ee0-e073eac435a3 nodeName:}" failed. No retries permitted until 2026-07-26 15:50:41.361618551 +0000 UTC m=+1925.651450667 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4059dd2e-1553-47fd-9ee0-e073eac435a3-maas-api-tls") pod "maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs" (UID: "4059dd2e-1553-47fd-9ee0-e073eac435a3") : secret "maas-api-e2e-mt-4debde71-serving-cert" not found
Jul 26 15:50:40.402418 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.402380    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn"]
Jul 26 15:50:40.413663 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.413617    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn"
Jul 26 15:50:40.416027 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.415999    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-4debde71-dockercfg-rtg5n\""
Jul 26 15:50:40.416188 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.415991    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-4debde71\""
Jul 26 15:50:40.416701 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.416671    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn"]
Jul 26 15:50:40.465178 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.465139    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv"]
Jul 26 15:50:40.468753 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.468729    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv"
Jul 26 15:50:40.476037 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.476010    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv"]
Jul 26 15:50:40.560706 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.560674    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:40.563922 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.563893    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjbj4\" (UniqueName: \"kubernetes.io/projected/95efc735-e4f7-4806-af55-6d2d2c219d3c-kube-api-access-wjbj4\") pod \"payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn\" (UID: \"95efc735-e4f7-4806-af55-6d2d2c219d3c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn"
Jul 26 15:50:40.564034 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.563967    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8890340-62cd-45eb-ae2a-1ba52c1f58e8-plugins-config-volume\") pod \"payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv\" (UID: \"b8890340-62cd-45eb-ae2a-1ba52c1f58e8\") " pod="openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv"
Jul 26 15:50:40.564088 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.564042    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95efc735-e4f7-4806-af55-6d2d2c219d3c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn\" (UID: \"95efc735-e4f7-4806-af55-6d2d2c219d3c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn"
Jul 26 15:50:40.564088 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.564058    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz4xs\" (UniqueName: \"kubernetes.io/projected/b8890340-62cd-45eb-ae2a-1ba52c1f58e8-kube-api-access-bz4xs\") pod \"payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv\" (UID: \"b8890340-62cd-45eb-ae2a-1ba52c1f58e8\") " pod="openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv"
Jul 26 15:50:40.665315 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.665279    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjbj4\" (UniqueName: \"kubernetes.io/projected/95efc735-e4f7-4806-af55-6d2d2c219d3c-kube-api-access-wjbj4\") pod \"payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn\" (UID: \"95efc735-e4f7-4806-af55-6d2d2c219d3c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn"
Jul 26 15:50:40.665507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.665347    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8890340-62cd-45eb-ae2a-1ba52c1f58e8-plugins-config-volume\") pod \"payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv\" (UID: \"b8890340-62cd-45eb-ae2a-1ba52c1f58e8\") " pod="openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv"
Jul 26 15:50:40.665507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.665402    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95efc735-e4f7-4806-af55-6d2d2c219d3c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn\" (UID: \"95efc735-e4f7-4806-af55-6d2d2c219d3c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn"
Jul 26 15:50:40.665507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.665418    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz4xs\" (UniqueName: \"kubernetes.io/projected/b8890340-62cd-45eb-ae2a-1ba52c1f58e8-kube-api-access-bz4xs\") pod \"payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv\" (UID: \"b8890340-62cd-45eb-ae2a-1ba52c1f58e8\") " pod="openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv"
Jul 26 15:50:40.666357 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.666334    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95efc735-e4f7-4806-af55-6d2d2c219d3c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn\" (UID: \"95efc735-e4f7-4806-af55-6d2d2c219d3c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn"
Jul 26 15:50:40.666411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.666334    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8890340-62cd-45eb-ae2a-1ba52c1f58e8-plugins-config-volume\") pod \"payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv\" (UID: \"b8890340-62cd-45eb-ae2a-1ba52c1f58e8\") " pod="openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv"
Jul 26 15:50:40.672766 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.672740    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjbj4\" (UniqueName: \"kubernetes.io/projected/95efc735-e4f7-4806-af55-6d2d2c219d3c-kube-api-access-wjbj4\") pod \"payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn\" (UID: \"95efc735-e4f7-4806-af55-6d2d2c219d3c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn"
Jul 26 15:50:40.672890 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.672825    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz4xs\" (UniqueName: \"kubernetes.io/projected/b8890340-62cd-45eb-ae2a-1ba52c1f58e8-kube-api-access-bz4xs\") pod \"payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv\" (UID: \"b8890340-62cd-45eb-ae2a-1ba52c1f58e8\") " pod="openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv"
Jul 26 15:50:40.726264 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.726186    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn"
Jul 26 15:50:40.781932 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.781874    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv"
Jul 26 15:50:40.881840 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.881355    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn"]
Jul 26 15:50:40.883874 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:50:40.883839    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95efc735_e4f7_4806_af55_6d2d2c219d3c.slice/crio-bc74d553610156b2fd695741df5ddf0877244bff1f68d587e30e932613f58970 WatchSource:0}: Error finding container bc74d553610156b2fd695741df5ddf0877244bff1f68d587e30e932613f58970: Status 404 returned error can't find the container with id bc74d553610156b2fd695741df5ddf0877244bff1f68d587e30e932613f58970
Jul 26 15:50:40.942137 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:40.942100    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv"]
Jul 26 15:50:40.944765 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:50:40.944732    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8890340_62cd_45eb_ae2a_1ba52c1f58e8.slice/crio-10479914dbb49ec65409ac0461e46ebc29934c8f76288ff2afa53deef7a356bb WatchSource:0}: Error finding container 10479914dbb49ec65409ac0461e46ebc29934c8f76288ff2afa53deef7a356bb: Status 404 returned error can't find the container with id 10479914dbb49ec65409ac0461e46ebc29934c8f76288ff2afa53deef7a356bb
Jul 26 15:50:41.373213 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:41.373172    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4059dd2e-1553-47fd-9ee0-e073eac435a3-maas-api-tls\") pod \"maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs\" (UID: \"4059dd2e-1553-47fd-9ee0-e073eac435a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"
Jul 26 15:50:41.375793 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:41.375768    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4059dd2e-1553-47fd-9ee0-e073eac435a3-maas-api-tls\") pod \"maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs\" (UID: \"4059dd2e-1553-47fd-9ee0-e073eac435a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"
Jul 26 15:50:41.443842 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:41.443805    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"
Jul 26 15:50:41.583172 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:41.583149    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"]
Jul 26 15:50:41.585057 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:50:41.585030    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4059dd2e_1553_47fd_9ee0_e073eac435a3.slice/crio-e4498431af2d6443034102cb589dd75e57af7656f33fddb500a364305ac06223 WatchSource:0}: Error finding container e4498431af2d6443034102cb589dd75e57af7656f33fddb500a364305ac06223: Status 404 returned error can't find the container with id e4498431af2d6443034102cb589dd75e57af7656f33fddb500a364305ac06223
Jul 26 15:50:41.685695 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:41.685584    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn" event={"ID":"95efc735-e4f7-4806-af55-6d2d2c219d3c","Type":"ContainerStarted","Data":"11afc0fbbb95024f77af7ab4543f0d033c723e2672bfeef6440ae23f85cec226"}
Jul 26 15:50:41.685695 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:41.685656    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn" event={"ID":"95efc735-e4f7-4806-af55-6d2d2c219d3c","Type":"ContainerStarted","Data":"bc74d553610156b2fd695741df5ddf0877244bff1f68d587e30e932613f58970"}
Jul 26 15:50:41.685695 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:41.685687    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn"
Jul 26 15:50:41.686802 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:41.686779    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs" event={"ID":"4059dd2e-1553-47fd-9ee0-e073eac435a3","Type":"ContainerStarted","Data":"e4498431af2d6443034102cb589dd75e57af7656f33fddb500a364305ac06223"}
Jul 26 15:50:41.688219 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:41.688180    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv" event={"ID":"b8890340-62cd-45eb-ae2a-1ba52c1f58e8","Type":"ContainerStarted","Data":"92523a5595e30b20965c39e8378ec4325f7fbe5a6b3030b06cc364d3a103ff1b"}
Jul 26 15:50:41.688318 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:41.688250    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv" event={"ID":"b8890340-62cd-45eb-ae2a-1ba52c1f58e8","Type":"ContainerStarted","Data":"10479914dbb49ec65409ac0461e46ebc29934c8f76288ff2afa53deef7a356bb"}
Jul 26 15:50:41.688318 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:41.688280    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv"
Jul 26 15:50:41.701576 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:41.701513    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn" podStartSLOduration=1.701493151 podStartE2EDuration="1.701493151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:50:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:50:41.699426663 +0000 UTC m=+1925.989258793" watchObservedRunningTime="2026-07-26 15:50:41.701493151 +0000 UTC m=+1925.991325271"
Jul 26 15:50:41.714054 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:41.714003    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv" podStartSLOduration=1.7139894500000001 podStartE2EDuration="1.71398945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:50:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:50:41.712775943 +0000 UTC m=+1926.002608062" watchObservedRunningTime="2026-07-26 15:50:41.71398945 +0000 UTC m=+1926.003821569"
Jul 26 15:50:42.694851 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:42.694815    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs" event={"ID":"4059dd2e-1553-47fd-9ee0-e073eac435a3","Type":"ContainerStarted","Data":"d9cd8386f9e41d5a99702d16b02c752c419ea2adf4541418e83fe5e96e6997b7"}
Jul 26 15:50:42.711807 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:42.711750    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs" podStartSLOduration=3.237354196 podStartE2EDuration="3.711731116s" totalImagesPullingTime="474.37692ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:50:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:50:42.709016974 +0000 UTC m=+1926.998849092" watchObservedRunningTime="2026-07-26 15:50:42.711731116 +0000 UTC m=+1927.001563235"
Jul 26 15:50:43.700463 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:43.700430    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"
Jul 26 15:50:44.038763 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:44.038661    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:44.675858 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:44.675821    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:44.773734 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:44.773699    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:45.116966 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:45.116924    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:49.711247 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:49.711198    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"
Jul 26 15:50:50.927426 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:50.927387    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:51.025439 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:51.025399    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:50:52.696880 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:52.696827    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn" podUID="95efc735-e4f7-4806-af55-6d2d2c219d3c" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.53:9004: connect: connection refused"
Jul 26 15:50:52.697667 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:50:52.697647    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv"
Jul 26 15:51:01.412118 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.412075    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"]
Jul 26 15:51:01.412626 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.412373    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs" podUID="4059dd2e-1553-47fd-9ee0-e073eac435a3" containerName="maas-api" containerID="cri-o://d9cd8386f9e41d5a99702d16b02c752c419ea2adf4541418e83fe5e96e6997b7" gracePeriod=30
Jul 26 15:51:01.439981 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.439950    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:01.465600 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.465565    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv"]
Jul 26 15:51:01.465857 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.465833    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv" podUID="b8890340-62cd-45eb-ae2a-1ba52c1f58e8" containerName="payload-processing" containerID="cri-o://92523a5595e30b20965c39e8378ec4325f7fbe5a6b3030b06cc364d3a103ff1b" gracePeriod=30
Jul 26 15:51:01.472184 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.472140    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn"]
Jul 26 15:51:01.472460 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.472436    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn" podUID="95efc735-e4f7-4806-af55-6d2d2c219d3c" containerName="payload-pre-processing" containerID="cri-o://11afc0fbbb95024f77af7ab4543f0d033c723e2672bfeef6440ae23f85cec226" gracePeriod=30
Jul 26 15:51:01.472989 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.472859    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn"
Jul 26 15:51:01.503129 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.503104    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:01.568134 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.568100    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:01.749663 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.749635    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"
Jul 26 15:51:01.753872 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.753846    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv"
Jul 26 15:51:01.776263 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.776238    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn"
Jul 26 15:51:01.808662 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.808622    2559 generic.go:356] "Generic (PLEG): container finished" podID="95efc735-e4f7-4806-af55-6d2d2c219d3c" containerID="11afc0fbbb95024f77af7ab4543f0d033c723e2672bfeef6440ae23f85cec226" exitCode=0
Jul 26 15:51:01.808873 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.808703    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn"
Jul 26 15:51:01.808873 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.808705    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn" event={"ID":"95efc735-e4f7-4806-af55-6d2d2c219d3c","Type":"ContainerDied","Data":"11afc0fbbb95024f77af7ab4543f0d033c723e2672bfeef6440ae23f85cec226"}
Jul 26 15:51:01.809094 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.809069    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn" event={"ID":"95efc735-e4f7-4806-af55-6d2d2c219d3c","Type":"ContainerDied","Data":"bc74d553610156b2fd695741df5ddf0877244bff1f68d587e30e932613f58970"}
Jul 26 15:51:01.809292 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.809104    2559 scope.go:117] "RemoveContainer" containerID="11afc0fbbb95024f77af7ab4543f0d033c723e2672bfeef6440ae23f85cec226"
Jul 26 15:51:01.810282 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.810156    2559 generic.go:356] "Generic (PLEG): container finished" podID="4059dd2e-1553-47fd-9ee0-e073eac435a3" containerID="d9cd8386f9e41d5a99702d16b02c752c419ea2adf4541418e83fe5e96e6997b7" exitCode=0
Jul 26 15:51:01.810282 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.810250    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs" event={"ID":"4059dd2e-1553-47fd-9ee0-e073eac435a3","Type":"ContainerDied","Data":"d9cd8386f9e41d5a99702d16b02c752c419ea2adf4541418e83fe5e96e6997b7"}
Jul 26 15:51:01.810428 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.810325    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs" event={"ID":"4059dd2e-1553-47fd-9ee0-e073eac435a3","Type":"ContainerDied","Data":"e4498431af2d6443034102cb589dd75e57af7656f33fddb500a364305ac06223"}
Jul 26 15:51:01.810428 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.810268    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"
Jul 26 15:51:01.811879 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.811859    2559 generic.go:356] "Generic (PLEG): container finished" podID="b8890340-62cd-45eb-ae2a-1ba52c1f58e8" containerID="92523a5595e30b20965c39e8378ec4325f7fbe5a6b3030b06cc364d3a103ff1b" exitCode=0
Jul 26 15:51:01.812004 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.811912    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv"
Jul 26 15:51:01.812004 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.811920    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv" event={"ID":"b8890340-62cd-45eb-ae2a-1ba52c1f58e8","Type":"ContainerDied","Data":"92523a5595e30b20965c39e8378ec4325f7fbe5a6b3030b06cc364d3a103ff1b"}
Jul 26 15:51:01.812004 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.811947    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv" event={"ID":"b8890340-62cd-45eb-ae2a-1ba52c1f58e8","Type":"ContainerDied","Data":"10479914dbb49ec65409ac0461e46ebc29934c8f76288ff2afa53deef7a356bb"}
Jul 26 15:51:01.819908 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.819890    2559 scope.go:117] "RemoveContainer" containerID="11afc0fbbb95024f77af7ab4543f0d033c723e2672bfeef6440ae23f85cec226"
Jul 26 15:51:01.820212 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:51:01.820179    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11afc0fbbb95024f77af7ab4543f0d033c723e2672bfeef6440ae23f85cec226\": container with ID starting with 11afc0fbbb95024f77af7ab4543f0d033c723e2672bfeef6440ae23f85cec226 not found: ID does not exist" containerID="11afc0fbbb95024f77af7ab4543f0d033c723e2672bfeef6440ae23f85cec226"
Jul 26 15:51:01.820286 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.820227    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11afc0fbbb95024f77af7ab4543f0d033c723e2672bfeef6440ae23f85cec226"} err="failed to get container status \"11afc0fbbb95024f77af7ab4543f0d033c723e2672bfeef6440ae23f85cec226\": rpc error: code = NotFound desc = could not find container \"11afc0fbbb95024f77af7ab4543f0d033c723e2672bfeef6440ae23f85cec226\": container with ID starting with 11afc0fbbb95024f77af7ab4543f0d033c723e2672bfeef6440ae23f85cec226 not found: ID does not exist"
Jul 26 15:51:01.820286 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.820251    2559 scope.go:117] "RemoveContainer" containerID="d9cd8386f9e41d5a99702d16b02c752c419ea2adf4541418e83fe5e96e6997b7"
Jul 26 15:51:01.828593 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.828571    2559 scope.go:117] "RemoveContainer" containerID="d9cd8386f9e41d5a99702d16b02c752c419ea2adf4541418e83fe5e96e6997b7"
Jul 26 15:51:01.828945 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:51:01.828917    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9cd8386f9e41d5a99702d16b02c752c419ea2adf4541418e83fe5e96e6997b7\": container with ID starting with d9cd8386f9e41d5a99702d16b02c752c419ea2adf4541418e83fe5e96e6997b7 not found: ID does not exist" containerID="d9cd8386f9e41d5a99702d16b02c752c419ea2adf4541418e83fe5e96e6997b7"
Jul 26 15:51:01.829029 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.828958    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9cd8386f9e41d5a99702d16b02c752c419ea2adf4541418e83fe5e96e6997b7"} err="failed to get container status \"d9cd8386f9e41d5a99702d16b02c752c419ea2adf4541418e83fe5e96e6997b7\": rpc error: code = NotFound desc = could not find container \"d9cd8386f9e41d5a99702d16b02c752c419ea2adf4541418e83fe5e96e6997b7\": container with ID starting with d9cd8386f9e41d5a99702d16b02c752c419ea2adf4541418e83fe5e96e6997b7 not found: ID does not exist"
Jul 26 15:51:01.829029 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.828985    2559 scope.go:117] "RemoveContainer" containerID="92523a5595e30b20965c39e8378ec4325f7fbe5a6b3030b06cc364d3a103ff1b"
Jul 26 15:51:01.837483 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.837459    2559 scope.go:117] "RemoveContainer" containerID="92523a5595e30b20965c39e8378ec4325f7fbe5a6b3030b06cc364d3a103ff1b"
Jul 26 15:51:01.837811 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:51:01.837790    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92523a5595e30b20965c39e8378ec4325f7fbe5a6b3030b06cc364d3a103ff1b\": container with ID starting with 92523a5595e30b20965c39e8378ec4325f7fbe5a6b3030b06cc364d3a103ff1b not found: ID does not exist" containerID="92523a5595e30b20965c39e8378ec4325f7fbe5a6b3030b06cc364d3a103ff1b"
Jul 26 15:51:01.837854 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.837820    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92523a5595e30b20965c39e8378ec4325f7fbe5a6b3030b06cc364d3a103ff1b"} err="failed to get container status \"92523a5595e30b20965c39e8378ec4325f7fbe5a6b3030b06cc364d3a103ff1b\": rpc error: code = NotFound desc = could not find container \"92523a5595e30b20965c39e8378ec4325f7fbe5a6b3030b06cc364d3a103ff1b\": container with ID starting with 92523a5595e30b20965c39e8378ec4325f7fbe5a6b3030b06cc364d3a103ff1b not found: ID does not exist"
Jul 26 15:51:01.863710 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.863682    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4059dd2e-1553-47fd-9ee0-e073eac435a3-maas-api-tls\") pod \"4059dd2e-1553-47fd-9ee0-e073eac435a3\" (UID: \"4059dd2e-1553-47fd-9ee0-e073eac435a3\") "
Jul 26 15:51:01.863851 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.863742    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhll2\" (UniqueName: \"kubernetes.io/projected/4059dd2e-1553-47fd-9ee0-e073eac435a3-kube-api-access-vhll2\") pod \"4059dd2e-1553-47fd-9ee0-e073eac435a3\" (UID: \"4059dd2e-1553-47fd-9ee0-e073eac435a3\") "
Jul 26 15:51:01.863851 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.863766    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4059dd2e-1553-47fd-9ee0-e073eac435a3-openshift-service-ca\") pod \"4059dd2e-1553-47fd-9ee0-e073eac435a3\" (UID: \"4059dd2e-1553-47fd-9ee0-e073eac435a3\") "
Jul 26 15:51:01.863928 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.863872    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjbj4\" (UniqueName: \"kubernetes.io/projected/95efc735-e4f7-4806-af55-6d2d2c219d3c-kube-api-access-wjbj4\") pod \"95efc735-e4f7-4806-af55-6d2d2c219d3c\" (UID: \"95efc735-e4f7-4806-af55-6d2d2c219d3c\") "
Jul 26 15:51:01.863928 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.863921    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95efc735-e4f7-4806-af55-6d2d2c219d3c-plugins-config-volume\") pod \"95efc735-e4f7-4806-af55-6d2d2c219d3c\" (UID: \"95efc735-e4f7-4806-af55-6d2d2c219d3c\") "
Jul 26 15:51:01.864004 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.863958    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bz4xs\" (UniqueName: \"kubernetes.io/projected/b8890340-62cd-45eb-ae2a-1ba52c1f58e8-kube-api-access-bz4xs\") pod \"b8890340-62cd-45eb-ae2a-1ba52c1f58e8\" (UID: \"b8890340-62cd-45eb-ae2a-1ba52c1f58e8\") "
Jul 26 15:51:01.864056 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.864012    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8890340-62cd-45eb-ae2a-1ba52c1f58e8-plugins-config-volume\") pod \"b8890340-62cd-45eb-ae2a-1ba52c1f58e8\" (UID: \"b8890340-62cd-45eb-ae2a-1ba52c1f58e8\") "
Jul 26 15:51:01.864129 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.864107    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4059dd2e-1553-47fd-9ee0-e073eac435a3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4059dd2e-1553-47fd-9ee0-e073eac435a3" (UID: "4059dd2e-1553-47fd-9ee0-e073eac435a3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:51:01.864402 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.864362    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95efc735-e4f7-4806-af55-6d2d2c219d3c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "95efc735-e4f7-4806-af55-6d2d2c219d3c" (UID: "95efc735-e4f7-4806-af55-6d2d2c219d3c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:51:01.864541 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.864412    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4059dd2e-1553-47fd-9ee0-e073eac435a3-openshift-service-ca\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:51:01.864541 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.864437    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8890340-62cd-45eb-ae2a-1ba52c1f58e8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8890340-62cd-45eb-ae2a-1ba52c1f58e8" (UID: "b8890340-62cd-45eb-ae2a-1ba52c1f58e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:51:01.865932 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.865907    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4059dd2e-1553-47fd-9ee0-e073eac435a3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4059dd2e-1553-47fd-9ee0-e073eac435a3" (UID: "4059dd2e-1553-47fd-9ee0-e073eac435a3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:51:01.866035 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.865945    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4059dd2e-1553-47fd-9ee0-e073eac435a3-kube-api-access-vhll2" (OuterVolumeSpecName: "kube-api-access-vhll2") pod "4059dd2e-1553-47fd-9ee0-e073eac435a3" (UID: "4059dd2e-1553-47fd-9ee0-e073eac435a3"). InnerVolumeSpecName "kube-api-access-vhll2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:51:01.866360 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.866334    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95efc735-e4f7-4806-af55-6d2d2c219d3c-kube-api-access-wjbj4" (OuterVolumeSpecName: "kube-api-access-wjbj4") pod "95efc735-e4f7-4806-af55-6d2d2c219d3c" (UID: "95efc735-e4f7-4806-af55-6d2d2c219d3c"). InnerVolumeSpecName "kube-api-access-wjbj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:51:01.866360 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.866348    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8890340-62cd-45eb-ae2a-1ba52c1f58e8-kube-api-access-bz4xs" (OuterVolumeSpecName: "kube-api-access-bz4xs") pod "b8890340-62cd-45eb-ae2a-1ba52c1f58e8" (UID: "b8890340-62cd-45eb-ae2a-1ba52c1f58e8"). InnerVolumeSpecName "kube-api-access-bz4xs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:51:01.965809 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.965716    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4059dd2e-1553-47fd-9ee0-e073eac435a3-maas-api-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:51:01.965809 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.965748    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhll2\" (UniqueName: \"kubernetes.io/projected/4059dd2e-1553-47fd-9ee0-e073eac435a3-kube-api-access-vhll2\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:51:01.965809 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.965760    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjbj4\" (UniqueName: \"kubernetes.io/projected/95efc735-e4f7-4806-af55-6d2d2c219d3c-kube-api-access-wjbj4\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:51:01.965809 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.965768    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95efc735-e4f7-4806-af55-6d2d2c219d3c-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:51:01.965809 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.965778    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz4xs\" (UniqueName: \"kubernetes.io/projected/b8890340-62cd-45eb-ae2a-1ba52c1f58e8-kube-api-access-bz4xs\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:51:01.965809 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:01.965786    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8890340-62cd-45eb-ae2a-1ba52c1f58e8-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:51:02.130571 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:02.130541    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn"]
Jul 26 15:51:02.134212 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:02.134169    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4debde71-869d8b64-4nsbn"]
Jul 26 15:51:02.142275 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:02.142245    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"]
Jul 26 15:51:02.145750 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:02.145726    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4debde71-fb7c475f6-f5ghs"]
Jul 26 15:51:02.154092 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:02.154064    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv"]
Jul 26 15:51:02.159125 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:02.159102    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4debde71-77946d6f66-n4kfv"]
Jul 26 15:51:02.194706 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:02.194671    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4059dd2e-1553-47fd-9ee0-e073eac435a3" path="/var/lib/kubelet/pods/4059dd2e-1553-47fd-9ee0-e073eac435a3/volumes"
Jul 26 15:51:02.195026 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:02.195013    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95efc735-e4f7-4806-af55-6d2d2c219d3c" path="/var/lib/kubelet/pods/95efc735-e4f7-4806-af55-6d2d2c219d3c/volumes"
Jul 26 15:51:02.195338 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:02.195326    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8890340-62cd-45eb-ae2a-1ba52c1f58e8" path="/var/lib/kubelet/pods/b8890340-62cd-45eb-ae2a-1ba52c1f58e8/volumes"
Jul 26 15:51:06.363839 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:06.363788    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:06.742622 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:06.742532    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:12.707374 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:12.707339    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:12.807335 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:12.807303    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:12.961422 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:12.961333    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:13.192379 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:13.192344    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:13.329044 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:13.328992    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:13.420148 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:13.420094    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:13.534089 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:13.534054    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:19.155841 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:19.155799    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:19.226391 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:19.226359    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:19.320389 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:19.320349    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:19.432128 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:19.432039    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:19.563101 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:19.563065    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:25.240776 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:25.240733    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:25.301362 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:25.301329    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:32.030684 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:32.030591    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:32.338680 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:32.338643    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:32.947660 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:32.947626    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:34.811234 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:34.811161    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:34.911015 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:34.910982    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:35.013098 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.013064    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"]
Jul 26 15:51:35.013587 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.013570    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4059dd2e-1553-47fd-9ee0-e073eac435a3" containerName="maas-api"
Jul 26 15:51:35.013587 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.013586    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4059dd2e-1553-47fd-9ee0-e073eac435a3" containerName="maas-api"
Jul 26 15:51:35.013726 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.013606    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95efc735-e4f7-4806-af55-6d2d2c219d3c" containerName="payload-pre-processing"
Jul 26 15:51:35.013726 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.013611    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="95efc735-e4f7-4806-af55-6d2d2c219d3c" containerName="payload-pre-processing"
Jul 26 15:51:35.013726 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.013620    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8890340-62cd-45eb-ae2a-1ba52c1f58e8" containerName="payload-processing"
Jul 26 15:51:35.013726 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.013626    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8890340-62cd-45eb-ae2a-1ba52c1f58e8" containerName="payload-processing"
Jul 26 15:51:35.013726 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.013687    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="95efc735-e4f7-4806-af55-6d2d2c219d3c" containerName="payload-pre-processing"
Jul 26 15:51:35.013726 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.013695    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4059dd2e-1553-47fd-9ee0-e073eac435a3" containerName="maas-api"
Jul 26 15:51:35.013726 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.013704    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8890340-62cd-45eb-ae2a-1ba52c1f58e8" containerName="payload-processing"
Jul 26 15:51:35.016929 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.016908    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"
Jul 26 15:51:35.018810 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.018789    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-35feb3ab-serving-cert\""
Jul 26 15:51:35.026123 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.026095    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"]
Jul 26 15:51:35.050401 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.050368    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:35.087030 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.086931    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rxhn\" (UniqueName: \"kubernetes.io/projected/e0d0a312-084b-4402-8867-2be30a7ff341-kube-api-access-9rxhn\") pod \"maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7\" (UID: \"e0d0a312-084b-4402-8867-2be30a7ff341\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"
Jul 26 15:51:35.087030 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.086973    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d0a312-084b-4402-8867-2be30a7ff341-openshift-service-ca\") pod \"maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7\" (UID: \"e0d0a312-084b-4402-8867-2be30a7ff341\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"
Jul 26 15:51:35.087265 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.087032    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0d0a312-084b-4402-8867-2be30a7ff341-maas-api-tls\") pod \"maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7\" (UID: \"e0d0a312-084b-4402-8867-2be30a7ff341\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"
Jul 26 15:51:35.188028 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.187989    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rxhn\" (UniqueName: \"kubernetes.io/projected/e0d0a312-084b-4402-8867-2be30a7ff341-kube-api-access-9rxhn\") pod \"maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7\" (UID: \"e0d0a312-084b-4402-8867-2be30a7ff341\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"
Jul 26 15:51:35.188028 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.188031    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d0a312-084b-4402-8867-2be30a7ff341-openshift-service-ca\") pod \"maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7\" (UID: \"e0d0a312-084b-4402-8867-2be30a7ff341\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"
Jul 26 15:51:35.188347 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.188209    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0d0a312-084b-4402-8867-2be30a7ff341-maas-api-tls\") pod \"maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7\" (UID: \"e0d0a312-084b-4402-8867-2be30a7ff341\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"
Jul 26 15:51:35.188402 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:51:35.188355    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-serving-cert: secret "maas-api-e2e-mt-35feb3ab-serving-cert" not found
Jul 26 15:51:35.188448 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:51:35.188438    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0d0a312-084b-4402-8867-2be30a7ff341-maas-api-tls podName:e0d0a312-084b-4402-8867-2be30a7ff341 nodeName:}" failed. No retries permitted until 2026-07-26 15:51:35.688416651 +0000 UTC m=+1979.978248763 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e0d0a312-084b-4402-8867-2be30a7ff341-maas-api-tls") pod "maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7" (UID: "e0d0a312-084b-4402-8867-2be30a7ff341") : secret "maas-api-e2e-mt-35feb3ab-serving-cert" not found
Jul 26 15:51:35.188722 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.188705    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d0a312-084b-4402-8867-2be30a7ff341-openshift-service-ca\") pod \"maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7\" (UID: \"e0d0a312-084b-4402-8867-2be30a7ff341\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"
Jul 26 15:51:35.199387 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.199352    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rxhn\" (UniqueName: \"kubernetes.io/projected/e0d0a312-084b-4402-8867-2be30a7ff341-kube-api-access-9rxhn\") pod \"maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7\" (UID: \"e0d0a312-084b-4402-8867-2be30a7ff341\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"
Jul 26 15:51:35.390432 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.390318    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:35.600889 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.600859    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:35.692035 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.691932    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0d0a312-084b-4402-8867-2be30a7ff341-maas-api-tls\") pod \"maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7\" (UID: \"e0d0a312-084b-4402-8867-2be30a7ff341\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"
Jul 26 15:51:35.694388 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.694358    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0d0a312-084b-4402-8867-2be30a7ff341-maas-api-tls\") pod \"maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7\" (UID: \"e0d0a312-084b-4402-8867-2be30a7ff341\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"
Jul 26 15:51:35.700462 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.700431    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l"]
Jul 26 15:51:35.700606 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.700460    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:35.704248 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.704229    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l"
Jul 26 15:51:35.706191 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.706173    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-35feb3ab-dockercfg-b6277\""
Jul 26 15:51:35.706337 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.706320    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-35feb3ab\""
Jul 26 15:51:35.716490 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.716465    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l"]
Jul 26 15:51:35.747954 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.747918    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg"]
Jul 26 15:51:35.751463 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.751446    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg"
Jul 26 15:51:35.761375 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.761346    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg"]
Jul 26 15:51:35.793150 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.793118    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b50c427-6f2a-474b-82ef-6210e2a3e429-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l\" (UID: \"6b50c427-6f2a-474b-82ef-6210e2a3e429\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l"
Jul 26 15:51:35.793150 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.793154    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfa227b5-39bd-4d32-aabe-35b32ddc7d94-plugins-config-volume\") pod \"payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg\" (UID: \"bfa227b5-39bd-4d32-aabe-35b32ddc7d94\") " pod="openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg"
Jul 26 15:51:35.793389 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.793296    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnkcp\" (UniqueName: \"kubernetes.io/projected/6b50c427-6f2a-474b-82ef-6210e2a3e429-kube-api-access-jnkcp\") pod \"payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l\" (UID: \"6b50c427-6f2a-474b-82ef-6210e2a3e429\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l"
Jul 26 15:51:35.793389 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.793359    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zxb7\" (UniqueName: \"kubernetes.io/projected/bfa227b5-39bd-4d32-aabe-35b32ddc7d94-kube-api-access-9zxb7\") pod \"payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg\" (UID: \"bfa227b5-39bd-4d32-aabe-35b32ddc7d94\") " pod="openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg"
Jul 26 15:51:35.893927 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.893892    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnkcp\" (UniqueName: \"kubernetes.io/projected/6b50c427-6f2a-474b-82ef-6210e2a3e429-kube-api-access-jnkcp\") pod \"payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l\" (UID: \"6b50c427-6f2a-474b-82ef-6210e2a3e429\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l"
Jul 26 15:51:35.894406 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.893964    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zxb7\" (UniqueName: \"kubernetes.io/projected/bfa227b5-39bd-4d32-aabe-35b32ddc7d94-kube-api-access-9zxb7\") pod \"payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg\" (UID: \"bfa227b5-39bd-4d32-aabe-35b32ddc7d94\") " pod="openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg"
Jul 26 15:51:35.894406 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.894113    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b50c427-6f2a-474b-82ef-6210e2a3e429-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l\" (UID: \"6b50c427-6f2a-474b-82ef-6210e2a3e429\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l"
Jul 26 15:51:35.894406 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.894150    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfa227b5-39bd-4d32-aabe-35b32ddc7d94-plugins-config-volume\") pod \"payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg\" (UID: \"bfa227b5-39bd-4d32-aabe-35b32ddc7d94\") " pod="openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg"
Jul 26 15:51:35.894838 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.894816    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b50c427-6f2a-474b-82ef-6210e2a3e429-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l\" (UID: \"6b50c427-6f2a-474b-82ef-6210e2a3e429\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l"
Jul 26 15:51:35.894970 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.894951    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfa227b5-39bd-4d32-aabe-35b32ddc7d94-plugins-config-volume\") pod \"payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg\" (UID: \"bfa227b5-39bd-4d32-aabe-35b32ddc7d94\") " pod="openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg"
Jul 26 15:51:35.901133 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.901106    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnkcp\" (UniqueName: \"kubernetes.io/projected/6b50c427-6f2a-474b-82ef-6210e2a3e429-kube-api-access-jnkcp\") pod \"payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l\" (UID: \"6b50c427-6f2a-474b-82ef-6210e2a3e429\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l"
Jul 26 15:51:35.901308 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.901114    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zxb7\" (UniqueName: \"kubernetes.io/projected/bfa227b5-39bd-4d32-aabe-35b32ddc7d94-kube-api-access-9zxb7\") pod \"payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg\" (UID: \"bfa227b5-39bd-4d32-aabe-35b32ddc7d94\") " pod="openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg"
Jul 26 15:51:35.928309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:35.928270    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"
Jul 26 15:51:36.015317 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:36.015270    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l"
Jul 26 15:51:36.062332 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:36.062295    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg"
Jul 26 15:51:36.072939 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:36.072870    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"]
Jul 26 15:51:36.078740 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:51:36.078693    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0d0a312_084b_4402_8867_2be30a7ff341.slice/crio-7a24307e20c4611e62127ee902b24bdf719ef9fb800798132459158eb458e30e WatchSource:0}: Error finding container 7a24307e20c4611e62127ee902b24bdf719ef9fb800798132459158eb458e30e: Status 404 returned error can't find the container with id 7a24307e20c4611e62127ee902b24bdf719ef9fb800798132459158eb458e30e
Jul 26 15:51:36.086187 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:36.086158    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 15:51:36.182850 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:36.182809    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l"]
Jul 26 15:51:36.184283 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:51:36.184253    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b50c427_6f2a_474b_82ef_6210e2a3e429.slice/crio-d7b3343edcaa808e87b69886cfaeb2c319952932d0a85b4e8bfc3b3b71d89d77 WatchSource:0}: Error finding container d7b3343edcaa808e87b69886cfaeb2c319952932d0a85b4e8bfc3b3b71d89d77: Status 404 returned error can't find the container with id d7b3343edcaa808e87b69886cfaeb2c319952932d0a85b4e8bfc3b3b71d89d77
Jul 26 15:51:36.232337 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:36.232310    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg"]
Jul 26 15:51:36.235095 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:51:36.235070    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfa227b5_39bd_4d32_aabe_35b32ddc7d94.slice/crio-7ecee2e32a03d470354ccc7dc7d52143d6f3331c52857d8aed13b7648cd03699 WatchSource:0}: Error finding container 7ecee2e32a03d470354ccc7dc7d52143d6f3331c52857d8aed13b7648cd03699: Status 404 returned error can't find the container with id 7ecee2e32a03d470354ccc7dc7d52143d6f3331c52857d8aed13b7648cd03699
Jul 26 15:51:36.994921 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:36.994858    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7" event={"ID":"e0d0a312-084b-4402-8867-2be30a7ff341","Type":"ContainerStarted","Data":"af260c1fbc71a0a618217045c432b27a7ec56077b240e3c5b5f168c0fcc311eb"}
Jul 26 15:51:36.995399 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:36.995276    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7" event={"ID":"e0d0a312-084b-4402-8867-2be30a7ff341","Type":"ContainerStarted","Data":"7a24307e20c4611e62127ee902b24bdf719ef9fb800798132459158eb458e30e"}
Jul 26 15:51:36.995399 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:36.995319    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"
Jul 26 15:51:36.996903 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:36.996875    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg" event={"ID":"bfa227b5-39bd-4d32-aabe-35b32ddc7d94","Type":"ContainerStarted","Data":"fcbadaad3377186aab3058b33d1af709de86db217de224d1dbf808564f19a4ec"}
Jul 26 15:51:36.997070 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:36.996924    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg" event={"ID":"bfa227b5-39bd-4d32-aabe-35b32ddc7d94","Type":"ContainerStarted","Data":"7ecee2e32a03d470354ccc7dc7d52143d6f3331c52857d8aed13b7648cd03699"}
Jul 26 15:51:36.997070 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:36.996956    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg"
Jul 26 15:51:36.998801 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:36.998777    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l" event={"ID":"6b50c427-6f2a-474b-82ef-6210e2a3e429","Type":"ContainerStarted","Data":"46e10b8827a29ba32789e908c8973ba4ff8df66051bf3766065d36dd133eef66"}
Jul 26 15:51:36.998949 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:36.998823    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l" event={"ID":"6b50c427-6f2a-474b-82ef-6210e2a3e429","Type":"ContainerStarted","Data":"d7b3343edcaa808e87b69886cfaeb2c319952932d0a85b4e8bfc3b3b71d89d77"}
Jul 26 15:51:36.998949 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:36.998850    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l"
Jul 26 15:51:37.011497 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:37.011440    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7" podStartSLOduration=2.636313518 podStartE2EDuration="3.011423929s" totalImagesPullingTime="375.110411ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:51:37.010320356 +0000 UTC m=+1981.300152476" watchObservedRunningTime="2026-07-26 15:51:37.011423929 +0000 UTC m=+1981.301256073"
Jul 26 15:51:37.027693 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:37.027636    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l" podStartSLOduration=2.027617301 podStartE2EDuration="2.027617301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:51:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:51:37.026363852 +0000 UTC m=+1981.316195976" watchObservedRunningTime="2026-07-26 15:51:37.027617301 +0000 UTC m=+1981.317449464"
Jul 26 15:51:37.044115 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:37.044051    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg" podStartSLOduration=2.044030005 podStartE2EDuration="2.044030005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:51:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:51:37.042144691 +0000 UTC m=+1981.331976810" watchObservedRunningTime="2026-07-26 15:51:37.044030005 +0000 UTC m=+1981.333862126"
Jul 26 15:51:39.288507 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:39.288475    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:51:43.010959 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:43.010928    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"
Jul 26 15:51:48.007303 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:48.007277    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l"
Jul 26 15:51:48.007802 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:48.007349    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg"
Jul 26 15:51:48.085621 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:51:48.085561    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:52:06.138065 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:06.138030    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:52:06.187715 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:06.187676    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:52:21.683034 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:21.682999    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:52:39.341540 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.341497    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"]
Jul 26 15:52:39.342130 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.341698    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7" podUID="e0d0a312-084b-4402-8867-2be30a7ff341" containerName="maas-api" containerID="cri-o://af260c1fbc71a0a618217045c432b27a7ec56077b240e3c5b5f168c0fcc311eb" gracePeriod=30
Jul 26 15:52:39.372920 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.372891    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:52:39.397025 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.396992    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg"]
Jul 26 15:52:39.397303 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.397272    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg" podUID="bfa227b5-39bd-4d32-aabe-35b32ddc7d94" containerName="payload-processing" containerID="cri-o://fcbadaad3377186aab3058b33d1af709de86db217de224d1dbf808564f19a4ec" gracePeriod=30
Jul 26 15:52:39.404520 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.404426    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l"]
Jul 26 15:52:39.404722 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.404646    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l" podUID="6b50c427-6f2a-474b-82ef-6210e2a3e429" containerName="payload-pre-processing" containerID="cri-o://46e10b8827a29ba32789e908c8973ba4ff8df66051bf3766065d36dd133eef66" gracePeriod=30
Jul 26 15:52:39.436948 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.436918    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:52:39.498764 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.498726    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:52:39.689652 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.689629    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg"
Jul 26 15:52:39.692754 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.692736    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"
Jul 26 15:52:39.704836 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.704812    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l"
Jul 26 15:52:39.802462 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.802425    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b50c427-6f2a-474b-82ef-6210e2a3e429-plugins-config-volume\") pod \"6b50c427-6f2a-474b-82ef-6210e2a3e429\" (UID: \"6b50c427-6f2a-474b-82ef-6210e2a3e429\") "
Jul 26 15:52:39.802670 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.802498    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnkcp\" (UniqueName: \"kubernetes.io/projected/6b50c427-6f2a-474b-82ef-6210e2a3e429-kube-api-access-jnkcp\") pod \"6b50c427-6f2a-474b-82ef-6210e2a3e429\" (UID: \"6b50c427-6f2a-474b-82ef-6210e2a3e429\") "
Jul 26 15:52:39.802670 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.802520    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zxb7\" (UniqueName: \"kubernetes.io/projected/bfa227b5-39bd-4d32-aabe-35b32ddc7d94-kube-api-access-9zxb7\") pod \"bfa227b5-39bd-4d32-aabe-35b32ddc7d94\" (UID: \"bfa227b5-39bd-4d32-aabe-35b32ddc7d94\") "
Jul 26 15:52:39.802670 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.802624    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfa227b5-39bd-4d32-aabe-35b32ddc7d94-plugins-config-volume\") pod \"bfa227b5-39bd-4d32-aabe-35b32ddc7d94\" (UID: \"bfa227b5-39bd-4d32-aabe-35b32ddc7d94\") "
Jul 26 15:52:39.802670 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.802649    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d0a312-084b-4402-8867-2be30a7ff341-openshift-service-ca\") pod \"e0d0a312-084b-4402-8867-2be30a7ff341\" (UID: \"e0d0a312-084b-4402-8867-2be30a7ff341\") "
Jul 26 15:52:39.802886 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.802685    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rxhn\" (UniqueName: \"kubernetes.io/projected/e0d0a312-084b-4402-8867-2be30a7ff341-kube-api-access-9rxhn\") pod \"e0d0a312-084b-4402-8867-2be30a7ff341\" (UID: \"e0d0a312-084b-4402-8867-2be30a7ff341\") "
Jul 26 15:52:39.802886 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.802715    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0d0a312-084b-4402-8867-2be30a7ff341-maas-api-tls\") pod \"e0d0a312-084b-4402-8867-2be30a7ff341\" (UID: \"e0d0a312-084b-4402-8867-2be30a7ff341\") "
Jul 26 15:52:39.802886 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.802818    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b50c427-6f2a-474b-82ef-6210e2a3e429-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6b50c427-6f2a-474b-82ef-6210e2a3e429" (UID: "6b50c427-6f2a-474b-82ef-6210e2a3e429"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:52:39.803076 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.803040    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfa227b5-39bd-4d32-aabe-35b32ddc7d94-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bfa227b5-39bd-4d32-aabe-35b32ddc7d94" (UID: "bfa227b5-39bd-4d32-aabe-35b32ddc7d94"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:52:39.803151 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.803061    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0d0a312-084b-4402-8867-2be30a7ff341-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e0d0a312-084b-4402-8867-2be30a7ff341" (UID: "e0d0a312-084b-4402-8867-2be30a7ff341"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:52:39.803289 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.803268    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfa227b5-39bd-4d32-aabe-35b32ddc7d94-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:52:39.803429 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.803295    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d0a312-084b-4402-8867-2be30a7ff341-openshift-service-ca\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:52:39.803429 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.803310    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b50c427-6f2a-474b-82ef-6210e2a3e429-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:52:39.804898 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.804871    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0d0a312-084b-4402-8867-2be30a7ff341-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e0d0a312-084b-4402-8867-2be30a7ff341" (UID: "e0d0a312-084b-4402-8867-2be30a7ff341"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:52:39.805008 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.804957    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0d0a312-084b-4402-8867-2be30a7ff341-kube-api-access-9rxhn" (OuterVolumeSpecName: "kube-api-access-9rxhn") pod "e0d0a312-084b-4402-8867-2be30a7ff341" (UID: "e0d0a312-084b-4402-8867-2be30a7ff341"). InnerVolumeSpecName "kube-api-access-9rxhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:52:39.805077 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.805062    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b50c427-6f2a-474b-82ef-6210e2a3e429-kube-api-access-jnkcp" (OuterVolumeSpecName: "kube-api-access-jnkcp") pod "6b50c427-6f2a-474b-82ef-6210e2a3e429" (UID: "6b50c427-6f2a-474b-82ef-6210e2a3e429"). InnerVolumeSpecName "kube-api-access-jnkcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:52:39.805119 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.805083    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfa227b5-39bd-4d32-aabe-35b32ddc7d94-kube-api-access-9zxb7" (OuterVolumeSpecName: "kube-api-access-9zxb7") pod "bfa227b5-39bd-4d32-aabe-35b32ddc7d94" (UID: "bfa227b5-39bd-4d32-aabe-35b32ddc7d94"). InnerVolumeSpecName "kube-api-access-9zxb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:52:39.904461 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.904363    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rxhn\" (UniqueName: \"kubernetes.io/projected/e0d0a312-084b-4402-8867-2be30a7ff341-kube-api-access-9rxhn\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:52:39.904461 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.904403    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0d0a312-084b-4402-8867-2be30a7ff341-maas-api-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:52:39.904461 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.904414    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnkcp\" (UniqueName: \"kubernetes.io/projected/6b50c427-6f2a-474b-82ef-6210e2a3e429-kube-api-access-jnkcp\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:52:39.904461 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:39.904423    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zxb7\" (UniqueName: \"kubernetes.io/projected/bfa227b5-39bd-4d32-aabe-35b32ddc7d94-kube-api-access-9zxb7\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:52:40.317831 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.317796    2559 generic.go:356] "Generic (PLEG): container finished" podID="bfa227b5-39bd-4d32-aabe-35b32ddc7d94" containerID="fcbadaad3377186aab3058b33d1af709de86db217de224d1dbf808564f19a4ec" exitCode=0
Jul 26 15:52:40.318032 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.317839    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg"
Jul 26 15:52:40.318032 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.317887    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg" event={"ID":"bfa227b5-39bd-4d32-aabe-35b32ddc7d94","Type":"ContainerDied","Data":"fcbadaad3377186aab3058b33d1af709de86db217de224d1dbf808564f19a4ec"}
Jul 26 15:52:40.318032 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.317945    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg" event={"ID":"bfa227b5-39bd-4d32-aabe-35b32ddc7d94","Type":"ContainerDied","Data":"7ecee2e32a03d470354ccc7dc7d52143d6f3331c52857d8aed13b7648cd03699"}
Jul 26 15:52:40.318032 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.317975    2559 scope.go:117] "RemoveContainer" containerID="fcbadaad3377186aab3058b33d1af709de86db217de224d1dbf808564f19a4ec"
Jul 26 15:52:40.319376 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.319356    2559 generic.go:356] "Generic (PLEG): container finished" podID="6b50c427-6f2a-474b-82ef-6210e2a3e429" containerID="46e10b8827a29ba32789e908c8973ba4ff8df66051bf3766065d36dd133eef66" exitCode=0
Jul 26 15:52:40.319477 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.319397    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l"
Jul 26 15:52:40.319477 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.319400    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l" event={"ID":"6b50c427-6f2a-474b-82ef-6210e2a3e429","Type":"ContainerDied","Data":"46e10b8827a29ba32789e908c8973ba4ff8df66051bf3766065d36dd133eef66"}
Jul 26 15:52:40.319477 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.319462    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l" event={"ID":"6b50c427-6f2a-474b-82ef-6210e2a3e429","Type":"ContainerDied","Data":"d7b3343edcaa808e87b69886cfaeb2c319952932d0a85b4e8bfc3b3b71d89d77"}
Jul 26 15:52:40.321012 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.320990    2559 generic.go:356] "Generic (PLEG): container finished" podID="e0d0a312-084b-4402-8867-2be30a7ff341" containerID="af260c1fbc71a0a618217045c432b27a7ec56077b240e3c5b5f168c0fcc311eb" exitCode=0
Jul 26 15:52:40.321132 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.321022    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7" event={"ID":"e0d0a312-084b-4402-8867-2be30a7ff341","Type":"ContainerDied","Data":"af260c1fbc71a0a618217045c432b27a7ec56077b240e3c5b5f168c0fcc311eb"}
Jul 26 15:52:40.321132 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.321058    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7" event={"ID":"e0d0a312-084b-4402-8867-2be30a7ff341","Type":"ContainerDied","Data":"7a24307e20c4611e62127ee902b24bdf719ef9fb800798132459158eb458e30e"}
Jul 26 15:52:40.321132 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.321023    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"
Jul 26 15:52:40.327701 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.327681    2559 scope.go:117] "RemoveContainer" containerID="fcbadaad3377186aab3058b33d1af709de86db217de224d1dbf808564f19a4ec"
Jul 26 15:52:40.327955 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:52:40.327935    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcbadaad3377186aab3058b33d1af709de86db217de224d1dbf808564f19a4ec\": container with ID starting with fcbadaad3377186aab3058b33d1af709de86db217de224d1dbf808564f19a4ec not found: ID does not exist" containerID="fcbadaad3377186aab3058b33d1af709de86db217de224d1dbf808564f19a4ec"
Jul 26 15:52:40.328013 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.327968    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcbadaad3377186aab3058b33d1af709de86db217de224d1dbf808564f19a4ec"} err="failed to get container status \"fcbadaad3377186aab3058b33d1af709de86db217de224d1dbf808564f19a4ec\": rpc error: code = NotFound desc = could not find container \"fcbadaad3377186aab3058b33d1af709de86db217de224d1dbf808564f19a4ec\": container with ID starting with fcbadaad3377186aab3058b33d1af709de86db217de224d1dbf808564f19a4ec not found: ID does not exist"
Jul 26 15:52:40.328013 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.327985    2559 scope.go:117] "RemoveContainer" containerID="46e10b8827a29ba32789e908c8973ba4ff8df66051bf3766065d36dd133eef66"
Jul 26 15:52:40.335568 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.335552    2559 scope.go:117] "RemoveContainer" containerID="46e10b8827a29ba32789e908c8973ba4ff8df66051bf3766065d36dd133eef66"
Jul 26 15:52:40.335802 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:52:40.335783    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46e10b8827a29ba32789e908c8973ba4ff8df66051bf3766065d36dd133eef66\": container with ID starting with 46e10b8827a29ba32789e908c8973ba4ff8df66051bf3766065d36dd133eef66 not found: ID does not exist" containerID="46e10b8827a29ba32789e908c8973ba4ff8df66051bf3766065d36dd133eef66"
Jul 26 15:52:40.335868 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.335810    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46e10b8827a29ba32789e908c8973ba4ff8df66051bf3766065d36dd133eef66"} err="failed to get container status \"46e10b8827a29ba32789e908c8973ba4ff8df66051bf3766065d36dd133eef66\": rpc error: code = NotFound desc = could not find container \"46e10b8827a29ba32789e908c8973ba4ff8df66051bf3766065d36dd133eef66\": container with ID starting with 46e10b8827a29ba32789e908c8973ba4ff8df66051bf3766065d36dd133eef66 not found: ID does not exist"
Jul 26 15:52:40.335868 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.335826    2559 scope.go:117] "RemoveContainer" containerID="af260c1fbc71a0a618217045c432b27a7ec56077b240e3c5b5f168c0fcc311eb"
Jul 26 15:52:40.339733 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.339711    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg"]
Jul 26 15:52:40.343460 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.343431    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-35feb3ab-6995588d7d-ffpwg"]
Jul 26 15:52:40.344877 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.344718    2559 scope.go:117] "RemoveContainer" containerID="af260c1fbc71a0a618217045c432b27a7ec56077b240e3c5b5f168c0fcc311eb"
Jul 26 15:52:40.345023 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:52:40.345007    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af260c1fbc71a0a618217045c432b27a7ec56077b240e3c5b5f168c0fcc311eb\": container with ID starting with af260c1fbc71a0a618217045c432b27a7ec56077b240e3c5b5f168c0fcc311eb not found: ID does not exist" containerID="af260c1fbc71a0a618217045c432b27a7ec56077b240e3c5b5f168c0fcc311eb"
Jul 26 15:52:40.345088 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.345030    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af260c1fbc71a0a618217045c432b27a7ec56077b240e3c5b5f168c0fcc311eb"} err="failed to get container status \"af260c1fbc71a0a618217045c432b27a7ec56077b240e3c5b5f168c0fcc311eb\": rpc error: code = NotFound desc = could not find container \"af260c1fbc71a0a618217045c432b27a7ec56077b240e3c5b5f168c0fcc311eb\": container with ID starting with af260c1fbc71a0a618217045c432b27a7ec56077b240e3c5b5f168c0fcc311eb not found: ID does not exist"
Jul 26 15:52:40.352854 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.352831    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l"]
Jul 26 15:52:40.358653 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.358630    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-35feb3ab-6d47d6d64d-8m67l"]
Jul 26 15:52:40.366901 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.366878    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"]
Jul 26 15:52:40.370107 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:40.370085    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-35feb3ab-9469c699c-4rdh7"]
Jul 26 15:52:42.193758 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:42.193728    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b50c427-6f2a-474b-82ef-6210e2a3e429" path="/var/lib/kubelet/pods/6b50c427-6f2a-474b-82ef-6210e2a3e429/volumes"
Jul 26 15:52:42.194136 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:42.194095    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfa227b5-39bd-4d32-aabe-35b32ddc7d94" path="/var/lib/kubelet/pods/bfa227b5-39bd-4d32-aabe-35b32ddc7d94/volumes"
Jul 26 15:52:42.194467 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:42.194453    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0d0a312-084b-4402-8867-2be30a7ff341" path="/var/lib/kubelet/pods/e0d0a312-084b-4402-8867-2be30a7ff341/volumes"
Jul 26 15:52:43.887044 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:43.887008    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:52:47.056051 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:47.055966    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:52:52.156860 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:52:52.156795    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:28.500637 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:28.500597    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:29.562609 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:29.562573    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:34.994325 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:34.994295    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:35.055859 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:35.055821    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:35.317723 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:35.317695    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:35.525779 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:35.525743    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:36.585450 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:36.585414    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:36.686358 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:36.686329    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:36.788218 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:36.788099    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:37.568336 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:37.568296    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:37.665993 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:37.665950    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:37.766044 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:37.766006    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:39.115539 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:39.115497    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:44.954282 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:44.954244    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:45.024820 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:45.024782    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:45.629054 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:45.629015    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:51.897833 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:51.897794    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:51.997522 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:51.997481    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:53:52.097225 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:53:52.097167    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:02.608928 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:02.608896    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:02.936056 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:02.935962    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:23.889716 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:23.889675    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:30.372225 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:30.372122    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:30.472155 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:30.472124    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:30.720565 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:30.720471    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:31.024776 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:31.024689    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:31.092942 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:31.092908    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:33.718281 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.718241    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:33.817954 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.817918    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:33.898770 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.898735    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"]
Jul 26 15:54:33.899134 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.899121    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfa227b5-39bd-4d32-aabe-35b32ddc7d94" containerName="payload-processing"
Jul 26 15:54:33.899182 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.899136    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfa227b5-39bd-4d32-aabe-35b32ddc7d94" containerName="payload-processing"
Jul 26 15:54:33.899182 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.899163    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b50c427-6f2a-474b-82ef-6210e2a3e429" containerName="payload-pre-processing"
Jul 26 15:54:33.899182 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.899171    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b50c427-6f2a-474b-82ef-6210e2a3e429" containerName="payload-pre-processing"
Jul 26 15:54:33.899182 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.899181    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d0a312-084b-4402-8867-2be30a7ff341" containerName="maas-api"
Jul 26 15:54:33.899343 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.899187    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d0a312-084b-4402-8867-2be30a7ff341" containerName="maas-api"
Jul 26 15:54:33.899343 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.899255    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b50c427-6f2a-474b-82ef-6210e2a3e429" containerName="payload-pre-processing"
Jul 26 15:54:33.899343 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.899265    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0d0a312-084b-4402-8867-2be30a7ff341" containerName="maas-api"
Jul 26 15:54:33.899343 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.899271    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfa227b5-39bd-4d32-aabe-35b32ddc7d94" containerName="payload-processing"
Jul 26 15:54:33.902459 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.902437    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"
Jul 26 15:54:33.904587 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.904555    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-ff0984-serving-cert\""
Jul 26 15:54:33.917248 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.917217    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"]
Jul 26 15:54:33.939597 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.939557    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:33.982014 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.981927    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6hr7\" (UniqueName: \"kubernetes.io/projected/052f9988-9747-47fe-9850-00fb4cf7a2a4-kube-api-access-m6hr7\") pod \"maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk\" (UID: \"052f9988-9747-47fe-9850-00fb4cf7a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"
Jul 26 15:54:33.982014 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.981987    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/052f9988-9747-47fe-9850-00fb4cf7a2a4-maas-api-tls\") pod \"maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk\" (UID: \"052f9988-9747-47fe-9850-00fb4cf7a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"
Jul 26 15:54:33.982014 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:33.982011    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/052f9988-9747-47fe-9850-00fb4cf7a2a4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk\" (UID: \"052f9988-9747-47fe-9850-00fb4cf7a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"
Jul 26 15:54:34.083265 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.083192    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6hr7\" (UniqueName: \"kubernetes.io/projected/052f9988-9747-47fe-9850-00fb4cf7a2a4-kube-api-access-m6hr7\") pod \"maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk\" (UID: \"052f9988-9747-47fe-9850-00fb4cf7a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"
Jul 26 15:54:34.083447 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.083290    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/052f9988-9747-47fe-9850-00fb4cf7a2a4-maas-api-tls\") pod \"maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk\" (UID: \"052f9988-9747-47fe-9850-00fb4cf7a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"
Jul 26 15:54:34.083447 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.083313    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/052f9988-9747-47fe-9850-00fb4cf7a2a4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk\" (UID: \"052f9988-9747-47fe-9850-00fb4cf7a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"
Jul 26 15:54:34.083447 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:54:34.083431    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-serving-cert: secret "maas-api-e2e-inf-a-ff0984-serving-cert" not found
Jul 26 15:54:34.083566 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:54:34.083498    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/052f9988-9747-47fe-9850-00fb4cf7a2a4-maas-api-tls podName:052f9988-9747-47fe-9850-00fb4cf7a2a4 nodeName:}" failed. No retries permitted until 2026-07-26 15:54:34.58348129 +0000 UTC m=+2158.873313399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/052f9988-9747-47fe-9850-00fb4cf7a2a4-maas-api-tls") pod "maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk" (UID: "052f9988-9747-47fe-9850-00fb4cf7a2a4") : secret "maas-api-e2e-inf-a-ff0984-serving-cert" not found
Jul 26 15:54:34.083892 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.083875    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/052f9988-9747-47fe-9850-00fb4cf7a2a4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk\" (UID: \"052f9988-9747-47fe-9850-00fb4cf7a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"
Jul 26 15:54:34.091808 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.091782    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6hr7\" (UniqueName: \"kubernetes.io/projected/052f9988-9747-47fe-9850-00fb4cf7a2a4-kube-api-access-m6hr7\") pod \"maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk\" (UID: \"052f9988-9747-47fe-9850-00fb4cf7a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"
Jul 26 15:54:34.148909 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.148865    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:34.400151 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.400107    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:34.507144 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.507109    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:34.507888 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.507864    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg"]
Jul 26 15:54:34.511626 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.511604    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg"
Jul 26 15:54:34.513676 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.513655    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-ff0984\""
Jul 26 15:54:34.513816 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.513655    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-ff0984-dockercfg-ccmrj\""
Jul 26 15:54:34.519972 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.519949    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg"]
Jul 26 15:54:34.543608 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.543575    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq"]
Jul 26 15:54:34.547190 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.547161    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq"
Jul 26 15:54:34.556179 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.556150    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq"]
Jul 26 15:54:34.588232 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.588157    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d08107c5-1b5e-4f84-8bd7-1c6153aa360f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg\" (UID: \"d08107c5-1b5e-4f84-8bd7-1c6153aa360f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg"
Jul 26 15:54:34.588419 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.588247    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/052f9988-9747-47fe-9850-00fb4cf7a2a4-maas-api-tls\") pod \"maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk\" (UID: \"052f9988-9747-47fe-9850-00fb4cf7a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"
Jul 26 15:54:34.588419 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.588282    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e15a33dd-590f-4670-bcad-c38e2d3db200-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq\" (UID: \"e15a33dd-590f-4670-bcad-c38e2d3db200\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq"
Jul 26 15:54:34.588419 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.588322    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkh2s\" (UniqueName: \"kubernetes.io/projected/e15a33dd-590f-4670-bcad-c38e2d3db200-kube-api-access-zkh2s\") pod \"payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq\" (UID: \"e15a33dd-590f-4670-bcad-c38e2d3db200\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq"
Jul 26 15:54:34.588419 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.588340    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twcp6\" (UniqueName: \"kubernetes.io/projected/d08107c5-1b5e-4f84-8bd7-1c6153aa360f-kube-api-access-twcp6\") pod \"payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg\" (UID: \"d08107c5-1b5e-4f84-8bd7-1c6153aa360f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg"
Jul 26 15:54:34.590691 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.590667    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/052f9988-9747-47fe-9850-00fb4cf7a2a4-maas-api-tls\") pod \"maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk\" (UID: \"052f9988-9747-47fe-9850-00fb4cf7a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"
Jul 26 15:54:34.689771 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.689670    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e15a33dd-590f-4670-bcad-c38e2d3db200-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq\" (UID: \"e15a33dd-590f-4670-bcad-c38e2d3db200\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq"
Jul 26 15:54:34.689771 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.689738    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkh2s\" (UniqueName: \"kubernetes.io/projected/e15a33dd-590f-4670-bcad-c38e2d3db200-kube-api-access-zkh2s\") pod \"payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq\" (UID: \"e15a33dd-590f-4670-bcad-c38e2d3db200\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq"
Jul 26 15:54:34.689771 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.689759    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twcp6\" (UniqueName: \"kubernetes.io/projected/d08107c5-1b5e-4f84-8bd7-1c6153aa360f-kube-api-access-twcp6\") pod \"payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg\" (UID: \"d08107c5-1b5e-4f84-8bd7-1c6153aa360f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg"
Jul 26 15:54:34.690001 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.689805    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d08107c5-1b5e-4f84-8bd7-1c6153aa360f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg\" (UID: \"d08107c5-1b5e-4f84-8bd7-1c6153aa360f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg"
Jul 26 15:54:34.690480 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.690462    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d08107c5-1b5e-4f84-8bd7-1c6153aa360f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg\" (UID: \"d08107c5-1b5e-4f84-8bd7-1c6153aa360f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg"
Jul 26 15:54:34.690530 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.690483    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e15a33dd-590f-4670-bcad-c38e2d3db200-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq\" (UID: \"e15a33dd-590f-4670-bcad-c38e2d3db200\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq"
Jul 26 15:54:34.698519 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.698482    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkh2s\" (UniqueName: \"kubernetes.io/projected/e15a33dd-590f-4670-bcad-c38e2d3db200-kube-api-access-zkh2s\") pod \"payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq\" (UID: \"e15a33dd-590f-4670-bcad-c38e2d3db200\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq"
Jul 26 15:54:34.698740 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.698721    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twcp6\" (UniqueName: \"kubernetes.io/projected/d08107c5-1b5e-4f84-8bd7-1c6153aa360f-kube-api-access-twcp6\") pod \"payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg\" (UID: \"d08107c5-1b5e-4f84-8bd7-1c6153aa360f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg"
Jul 26 15:54:34.813094 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.813054    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"
Jul 26 15:54:34.823525 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.823493    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg"
Jul 26 15:54:34.858422 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.858386    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq"
Jul 26 15:54:34.994392 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:34.994282    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"]
Jul 26 15:54:34.996460 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:54:34.996411    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod052f9988_9747_47fe_9850_00fb4cf7a2a4.slice/crio-b37142121345c5471841421e35c9b13131ead8ba3373d20df46d7ec5091e6c7e WatchSource:0}: Error finding container b37142121345c5471841421e35c9b13131ead8ba3373d20df46d7ec5091e6c7e: Status 404 returned error can't find the container with id b37142121345c5471841421e35c9b13131ead8ba3373d20df46d7ec5091e6c7e
Jul 26 15:54:35.017767 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:35.017720    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg"]
Jul 26 15:54:35.021436 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:54:35.021395    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd08107c5_1b5e_4f84_8bd7_1c6153aa360f.slice/crio-9e4e0d811abe2651c66dfbd336fc59c3d3236da29b15854073bee92132639331 WatchSource:0}: Error finding container 9e4e0d811abe2651c66dfbd336fc59c3d3236da29b15854073bee92132639331: Status 404 returned error can't find the container with id 9e4e0d811abe2651c66dfbd336fc59c3d3236da29b15854073bee92132639331
Jul 26 15:54:35.044615 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:35.044586    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq"]
Jul 26 15:54:35.046191 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:54:35.046166    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode15a33dd_590f_4670_bcad_c38e2d3db200.slice/crio-0be95aa221472ac1b51e76375e334dc383d6c9a23f4e5c9e609c06bb27981db7 WatchSource:0}: Error finding container 0be95aa221472ac1b51e76375e334dc383d6c9a23f4e5c9e609c06bb27981db7: Status 404 returned error can't find the container with id 0be95aa221472ac1b51e76375e334dc383d6c9a23f4e5c9e609c06bb27981db7
Jul 26 15:54:35.891664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:35.891616    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk" event={"ID":"052f9988-9747-47fe-9850-00fb4cf7a2a4","Type":"ContainerStarted","Data":"e10ec917ae2432a83ddbba7ab02705cb528351eef609b5fdf65ff72b7c1e1f46"}
Jul 26 15:54:35.892175 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:35.891693    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk" event={"ID":"052f9988-9747-47fe-9850-00fb4cf7a2a4","Type":"ContainerStarted","Data":"b37142121345c5471841421e35c9b13131ead8ba3373d20df46d7ec5091e6c7e"}
Jul 26 15:54:35.892175 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:35.891752    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"
Jul 26 15:54:35.893269 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:35.893236    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq" event={"ID":"e15a33dd-590f-4670-bcad-c38e2d3db200","Type":"ContainerStarted","Data":"342375ef3fade2d732b04b9e9793408b7093e2d08c66d59ebc73dc372e149e40"}
Jul 26 15:54:35.893411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:35.893283    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq" event={"ID":"e15a33dd-590f-4670-bcad-c38e2d3db200","Type":"ContainerStarted","Data":"0be95aa221472ac1b51e76375e334dc383d6c9a23f4e5c9e609c06bb27981db7"}
Jul 26 15:54:35.893411 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:35.893309    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq"
Jul 26 15:54:35.895011 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:35.894979    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg" event={"ID":"d08107c5-1b5e-4f84-8bd7-1c6153aa360f","Type":"ContainerStarted","Data":"0d7368d6cb7fb3d91bd8e21cb5d8e715a516be095dcd659c9c2ddf021783f988"}
Jul 26 15:54:35.895119 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:35.895023    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg" event={"ID":"d08107c5-1b5e-4f84-8bd7-1c6153aa360f","Type":"ContainerStarted","Data":"9e4e0d811abe2651c66dfbd336fc59c3d3236da29b15854073bee92132639331"}
Jul 26 15:54:35.895119 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:35.895060    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg"
Jul 26 15:54:35.908820 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:35.908771    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk" podStartSLOduration=2.396876357 podStartE2EDuration="2.908757405s" totalImagesPullingTime="511.881048ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:54:35.906895801 +0000 UTC m=+2160.196727921" watchObservedRunningTime="2026-07-26 15:54:35.908757405 +0000 UTC m=+2160.198589524"
Jul 26 15:54:35.923349 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:35.923264    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg" podStartSLOduration=1.923198325 podStartE2EDuration="1.923198325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:54:35.923002987 +0000 UTC m=+2160.212835107" watchObservedRunningTime="2026-07-26 15:54:35.923198325 +0000 UTC m=+2160.213030444"
Jul 26 15:54:35.939389 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:35.939336    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq" podStartSLOduration=1.939319207 podStartE2EDuration="1.939319207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:54:35.938135378 +0000 UTC m=+2160.227967498" watchObservedRunningTime="2026-07-26 15:54:35.939319207 +0000 UTC m=+2160.229151326"
Jul 26 15:54:38.207164 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:38.207116    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:38.534602 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:38.534477    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:38.828192 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:38.828154    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:39.671680 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:39.671648    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:41.768993 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:41.768954    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:41.843977 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:41.843941    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:41.845129 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:41.845101    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"]
Jul 26 15:54:41.848863 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:41.848841    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"
Jul 26 15:54:41.850700 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:41.850678    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-8c61fd-serving-cert\""
Jul 26 15:54:41.858907 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:41.858869    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"]
Jul 26 15:54:41.906118 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:41.906086    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"
Jul 26 15:54:41.907417 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:41.907392    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:41.963015 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:41.962985    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8974\" (UniqueName: \"kubernetes.io/projected/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-kube-api-access-h8974\") pod \"maas-api-e2e-inf-b-8c61fd-677478655c-mnntq\" (UID: \"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"
Jul 26 15:54:41.963227 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:41.963072    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-openshift-service-ca\") pod \"maas-api-e2e-inf-b-8c61fd-677478655c-mnntq\" (UID: \"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"
Jul 26 15:54:41.963227 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:41.963104    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-maas-api-tls\") pod \"maas-api-e2e-inf-b-8c61fd-677478655c-mnntq\" (UID: \"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"
Jul 26 15:54:42.064599 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.064558    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-maas-api-tls\") pod \"maas-api-e2e-inf-b-8c61fd-677478655c-mnntq\" (UID: \"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"
Jul 26 15:54:42.064798 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.064658    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8974\" (UniqueName: \"kubernetes.io/projected/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-kube-api-access-h8974\") pod \"maas-api-e2e-inf-b-8c61fd-677478655c-mnntq\" (UID: \"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"
Jul 26 15:54:42.064798 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:54:42.064714    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-serving-cert: secret "maas-api-e2e-inf-b-8c61fd-serving-cert" not found
Jul 26 15:54:42.064798 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.064747    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-openshift-service-ca\") pod \"maas-api-e2e-inf-b-8c61fd-677478655c-mnntq\" (UID: \"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"
Jul 26 15:54:42.064798 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:54:42.064775    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-maas-api-tls podName:dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6 nodeName:}" failed. No retries permitted until 2026-07-26 15:54:42.564758687 +0000 UTC m=+2166.854590783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-maas-api-tls") pod "maas-api-e2e-inf-b-8c61fd-677478655c-mnntq" (UID: "dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6") : secret "maas-api-e2e-inf-b-8c61fd-serving-cert" not found
Jul 26 15:54:42.065408 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.065387    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-openshift-service-ca\") pod \"maas-api-e2e-inf-b-8c61fd-677478655c-mnntq\" (UID: \"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"
Jul 26 15:54:42.073175 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.073151    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8974\" (UniqueName: \"kubernetes.io/projected/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-kube-api-access-h8974\") pod \"maas-api-e2e-inf-b-8c61fd-677478655c-mnntq\" (UID: \"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"
Jul 26 15:54:42.140958 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.140921    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:42.444557 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.444472    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:42.542174 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.542138    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:42.555789 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.555754    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9"]
Jul 26 15:54:42.559570 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.559544    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9"
Jul 26 15:54:42.563287 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.563257    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-8c61fd\""
Jul 26 15:54:42.563425 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.563309    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-8c61fd-dockercfg-98fnd\""
Jul 26 15:54:42.569194 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.569166    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-maas-api-tls\") pod \"maas-api-e2e-inf-b-8c61fd-677478655c-mnntq\" (UID: \"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"
Jul 26 15:54:42.572301 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.572268    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-maas-api-tls\") pod \"maas-api-e2e-inf-b-8c61fd-677478655c-mnntq\" (UID: \"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"
Jul 26 15:54:42.574983 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.574960    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9"]
Jul 26 15:54:42.670471 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.670439    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8c1b62f-a850-414f-8b29-3b7f27b21196-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9\" (UID: \"b8c1b62f-a850-414f-8b29-3b7f27b21196\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9"
Jul 26 15:54:42.670471 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.670475    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d7tw\" (UniqueName: \"kubernetes.io/projected/b8c1b62f-a850-414f-8b29-3b7f27b21196-kube-api-access-7d7tw\") pod \"payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9\" (UID: \"b8c1b62f-a850-414f-8b29-3b7f27b21196\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9"
Jul 26 15:54:42.762379 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.762281    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"
Jul 26 15:54:42.771639 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.771612    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8c1b62f-a850-414f-8b29-3b7f27b21196-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9\" (UID: \"b8c1b62f-a850-414f-8b29-3b7f27b21196\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9"
Jul 26 15:54:42.772040 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.771645    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d7tw\" (UniqueName: \"kubernetes.io/projected/b8c1b62f-a850-414f-8b29-3b7f27b21196-kube-api-access-7d7tw\") pod \"payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9\" (UID: \"b8c1b62f-a850-414f-8b29-3b7f27b21196\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9"
Jul 26 15:54:42.772516 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.772490    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8c1b62f-a850-414f-8b29-3b7f27b21196-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9\" (UID: \"b8c1b62f-a850-414f-8b29-3b7f27b21196\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9"
Jul 26 15:54:42.778854 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.778828    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d7tw\" (UniqueName: \"kubernetes.io/projected/b8c1b62f-a850-414f-8b29-3b7f27b21196-kube-api-access-7d7tw\") pod \"payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9\" (UID: \"b8c1b62f-a850-414f-8b29-3b7f27b21196\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9"
Jul 26 15:54:42.870723 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.870691    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9"
Jul 26 15:54:42.905514 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.905478    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"]
Jul 26 15:54:42.908416 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:54:42.908386    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc9680cd_038a_43f8_b5fb_e11e8f4d4ed6.slice/crio-64eb2250abf599af94b44448626ab0d44ed8246b45cb3e3220c1ef1263347cd7 WatchSource:0}: Error finding container 64eb2250abf599af94b44448626ab0d44ed8246b45cb3e3220c1ef1263347cd7: Status 404 returned error can't find the container with id 64eb2250abf599af94b44448626ab0d44ed8246b45cb3e3220c1ef1263347cd7
Jul 26 15:54:42.933121 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:42.933086    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq" event={"ID":"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6","Type":"ContainerStarted","Data":"64eb2250abf599af94b44448626ab0d44ed8246b45cb3e3220c1ef1263347cd7"}
Jul 26 15:54:43.023972 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:43.023917    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9"]
Jul 26 15:54:43.026197 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:54:43.026169    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8c1b62f_a850_414f_8b29_3b7f27b21196.slice/crio-047a450ae489570e5d9c4d3dfdba4bf70ea335c94c6d9819971ebccaf7a55a06 WatchSource:0}: Error finding container 047a450ae489570e5d9c4d3dfdba4bf70ea335c94c6d9819971ebccaf7a55a06: Status 404 returned error can't find the container with id 047a450ae489570e5d9c4d3dfdba4bf70ea335c94c6d9819971ebccaf7a55a06
Jul 26 15:54:43.939882 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:43.939835    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9" event={"ID":"b8c1b62f-a850-414f-8b29-3b7f27b21196","Type":"ContainerStarted","Data":"1656f2a997907a2d086d316a4592a5bd3b9c15c954e349b70fdd5edfc4172566"}
Jul 26 15:54:43.940376 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:43.939902    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9"
Jul 26 15:54:43.940376 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:43.939927    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9" event={"ID":"b8c1b62f-a850-414f-8b29-3b7f27b21196","Type":"ContainerStarted","Data":"047a450ae489570e5d9c4d3dfdba4bf70ea335c94c6d9819971ebccaf7a55a06"}
Jul 26 15:54:43.941344 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:43.941323    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq" event={"ID":"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6","Type":"ContainerStarted","Data":"f137dc163029d9d024cce31ab73b5944276a023460c0a9ff6151709c40d6074a"}
Jul 26 15:54:43.941426 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:43.941366    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"
Jul 26 15:54:43.958856 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:43.958784    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9" podStartSLOduration=1.958768208 podStartE2EDuration="1.958768208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:54:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:54:43.95663344 +0000 UTC m=+2168.246465568" watchObservedRunningTime="2026-07-26 15:54:43.958768208 +0000 UTC m=+2168.248600318"
Jul 26 15:54:43.973560 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:43.973507    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq" podStartSLOduration=2.631292563 podStartE2EDuration="2.973492094s" totalImagesPullingTime="342.199531ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:54:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:54:43.971618937 +0000 UTC m=+2168.261451055" watchObservedRunningTime="2026-07-26 15:54:43.973492094 +0000 UTC m=+2168.263324214"
Jul 26 15:54:46.271282 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:46.271247    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:54:46.902323 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:46.902223    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq"
Jul 26 15:54:46.902323 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:46.902302    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg"
Jul 26 15:54:49.952807 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:49.952777    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"
Jul 26 15:54:54.949108 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:54:54.949075    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9"
Jul 26 15:55:00.805005 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:55:00.804965    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:55:04.167737 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:55:04.167700    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:55:19.814606 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:55:19.814556    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:00.582938 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:00.582839    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:01.586152 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:01.586120    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:12.198564 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.198520    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"]
Jul 26 15:56:12.199066 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.198736    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk" podUID="052f9988-9747-47fe-9850-00fb4cf7a2a4" containerName="maas-api" containerID="cri-o://e10ec917ae2432a83ddbba7ab02705cb528351eef609b5fdf65ff72b7c1e1f46" gracePeriod=30
Jul 26 15:56:12.218934 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.218905    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:12.247287 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.247251    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq"]
Jul 26 15:56:12.247552 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.247515    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq" podUID="e15a33dd-590f-4670-bcad-c38e2d3db200" containerName="payload-processing" containerID="cri-o://342375ef3fade2d732b04b9e9793408b7093e2d08c66d59ebc73dc372e149e40" gracePeriod=30
Jul 26 15:56:12.260152 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.260118    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg"]
Jul 26 15:56:12.320070 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.320033    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:12.340671 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:56:12.340630    2559 configmap.go:193] Couldn't get configMap openshift-ingress/payload-processing-plugins-e2e-inf-a-ff0984: configmap "payload-processing-plugins-e2e-inf-a-ff0984" not found
Jul 26 15:56:12.340873 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:56:12.340809    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d08107c5-1b5e-4f84-8bd7-1c6153aa360f-plugins-config-volume podName:d08107c5-1b5e-4f84-8bd7-1c6153aa360f nodeName:}" failed. No retries permitted until 2026-07-26 15:56:12.840774682 +0000 UTC m=+2257.130606790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/d08107c5-1b5e-4f84-8bd7-1c6153aa360f-plugins-config-volume") pod "payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg" (UID: "d08107c5-1b5e-4f84-8bd7-1c6153aa360f") : configmap "payload-processing-plugins-e2e-inf-a-ff0984" not found
Jul 26 15:56:12.409337 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.409224    2559 generic.go:356] "Generic (PLEG): container finished" podID="052f9988-9747-47fe-9850-00fb4cf7a2a4" containerID="e10ec917ae2432a83ddbba7ab02705cb528351eef609b5fdf65ff72b7c1e1f46" exitCode=0
Jul 26 15:56:12.409337 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.409234    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk" event={"ID":"052f9988-9747-47fe-9850-00fb4cf7a2a4","Type":"ContainerDied","Data":"e10ec917ae2432a83ddbba7ab02705cb528351eef609b5fdf65ff72b7c1e1f46"}
Jul 26 15:56:12.410974 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.410949    2559 generic.go:356] "Generic (PLEG): container finished" podID="e15a33dd-590f-4670-bcad-c38e2d3db200" containerID="342375ef3fade2d732b04b9e9793408b7093e2d08c66d59ebc73dc372e149e40" exitCode=0
Jul 26 15:56:12.411111 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.411022    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq" event={"ID":"e15a33dd-590f-4670-bcad-c38e2d3db200","Type":"ContainerDied","Data":"342375ef3fade2d732b04b9e9793408b7093e2d08c66d59ebc73dc372e149e40"}
Jul 26 15:56:12.411376 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.411231    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg" podUID="d08107c5-1b5e-4f84-8bd7-1c6153aa360f" containerName="payload-pre-processing" containerID="cri-o://0d7368d6cb7fb3d91bd8e21cb5d8e715a516be095dcd659c9c2ddf021783f988" gracePeriod=30
Jul 26 15:56:12.420010 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.419971    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:12.534798 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.534768    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"
Jul 26 15:56:12.538711 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.538682    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq"
Jul 26 15:56:12.643509 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.643473    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6hr7\" (UniqueName: \"kubernetes.io/projected/052f9988-9747-47fe-9850-00fb4cf7a2a4-kube-api-access-m6hr7\") pod \"052f9988-9747-47fe-9850-00fb4cf7a2a4\" (UID: \"052f9988-9747-47fe-9850-00fb4cf7a2a4\") "
Jul 26 15:56:12.643715 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.643607    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e15a33dd-590f-4670-bcad-c38e2d3db200-plugins-config-volume\") pod \"e15a33dd-590f-4670-bcad-c38e2d3db200\" (UID: \"e15a33dd-590f-4670-bcad-c38e2d3db200\") "
Jul 26 15:56:12.643715 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.643677    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkh2s\" (UniqueName: \"kubernetes.io/projected/e15a33dd-590f-4670-bcad-c38e2d3db200-kube-api-access-zkh2s\") pod \"e15a33dd-590f-4670-bcad-c38e2d3db200\" (UID: \"e15a33dd-590f-4670-bcad-c38e2d3db200\") "
Jul 26 15:56:12.643715 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.643705    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/052f9988-9747-47fe-9850-00fb4cf7a2a4-openshift-service-ca\") pod \"052f9988-9747-47fe-9850-00fb4cf7a2a4\" (UID: \"052f9988-9747-47fe-9850-00fb4cf7a2a4\") "
Jul 26 15:56:12.643893 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.643751    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/052f9988-9747-47fe-9850-00fb4cf7a2a4-maas-api-tls\") pod \"052f9988-9747-47fe-9850-00fb4cf7a2a4\" (UID: \"052f9988-9747-47fe-9850-00fb4cf7a2a4\") "
Jul 26 15:56:12.644126 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.644084    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e15a33dd-590f-4670-bcad-c38e2d3db200-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e15a33dd-590f-4670-bcad-c38e2d3db200" (UID: "e15a33dd-590f-4670-bcad-c38e2d3db200"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:56:12.644231 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.644125    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/052f9988-9747-47fe-9850-00fb4cf7a2a4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "052f9988-9747-47fe-9850-00fb4cf7a2a4" (UID: "052f9988-9747-47fe-9850-00fb4cf7a2a4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:56:12.647771 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.647746    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e15a33dd-590f-4670-bcad-c38e2d3db200-kube-api-access-zkh2s" (OuterVolumeSpecName: "kube-api-access-zkh2s") pod "e15a33dd-590f-4670-bcad-c38e2d3db200" (UID: "e15a33dd-590f-4670-bcad-c38e2d3db200"). InnerVolumeSpecName "kube-api-access-zkh2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:56:12.647904 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.647887    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/052f9988-9747-47fe-9850-00fb4cf7a2a4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "052f9988-9747-47fe-9850-00fb4cf7a2a4" (UID: "052f9988-9747-47fe-9850-00fb4cf7a2a4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:56:12.647965 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.647917    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/052f9988-9747-47fe-9850-00fb4cf7a2a4-kube-api-access-m6hr7" (OuterVolumeSpecName: "kube-api-access-m6hr7") pod "052f9988-9747-47fe-9850-00fb4cf7a2a4" (UID: "052f9988-9747-47fe-9850-00fb4cf7a2a4"). InnerVolumeSpecName "kube-api-access-m6hr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:56:12.675986 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.675959    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg"
Jul 26 15:56:12.744773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.744677    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6hr7\" (UniqueName: \"kubernetes.io/projected/052f9988-9747-47fe-9850-00fb4cf7a2a4-kube-api-access-m6hr7\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:56:12.744773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.744711    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e15a33dd-590f-4670-bcad-c38e2d3db200-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:56:12.744773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.744720    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkh2s\" (UniqueName: \"kubernetes.io/projected/e15a33dd-590f-4670-bcad-c38e2d3db200-kube-api-access-zkh2s\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:56:12.744773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.744731    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/052f9988-9747-47fe-9850-00fb4cf7a2a4-openshift-service-ca\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:56:12.744773 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.744741    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/052f9988-9747-47fe-9850-00fb4cf7a2a4-maas-api-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:56:12.845983 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.845948    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twcp6\" (UniqueName: \"kubernetes.io/projected/d08107c5-1b5e-4f84-8bd7-1c6153aa360f-kube-api-access-twcp6\") pod \"d08107c5-1b5e-4f84-8bd7-1c6153aa360f\" (UID: \"d08107c5-1b5e-4f84-8bd7-1c6153aa360f\") "
Jul 26 15:56:12.846186 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.845997    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d08107c5-1b5e-4f84-8bd7-1c6153aa360f-plugins-config-volume\") pod \"d08107c5-1b5e-4f84-8bd7-1c6153aa360f\" (UID: \"d08107c5-1b5e-4f84-8bd7-1c6153aa360f\") "
Jul 26 15:56:12.846467 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.846438    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d08107c5-1b5e-4f84-8bd7-1c6153aa360f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d08107c5-1b5e-4f84-8bd7-1c6153aa360f" (UID: "d08107c5-1b5e-4f84-8bd7-1c6153aa360f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:56:12.848104 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.848075    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d08107c5-1b5e-4f84-8bd7-1c6153aa360f-kube-api-access-twcp6" (OuterVolumeSpecName: "kube-api-access-twcp6") pod "d08107c5-1b5e-4f84-8bd7-1c6153aa360f" (UID: "d08107c5-1b5e-4f84-8bd7-1c6153aa360f"). InnerVolumeSpecName "kube-api-access-twcp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:56:12.947792 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.947753    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twcp6\" (UniqueName: \"kubernetes.io/projected/d08107c5-1b5e-4f84-8bd7-1c6153aa360f-kube-api-access-twcp6\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:56:12.947792 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:12.947785    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d08107c5-1b5e-4f84-8bd7-1c6153aa360f-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:56:13.417650 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.417616    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"
Jul 26 15:56:13.417650 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.417642    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk" event={"ID":"052f9988-9747-47fe-9850-00fb4cf7a2a4","Type":"ContainerDied","Data":"b37142121345c5471841421e35c9b13131ead8ba3373d20df46d7ec5091e6c7e"}
Jul 26 15:56:13.418195 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.417704    2559 scope.go:117] "RemoveContainer" containerID="e10ec917ae2432a83ddbba7ab02705cb528351eef609b5fdf65ff72b7c1e1f46"
Jul 26 15:56:13.418992 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.418969    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq"
Jul 26 15:56:13.419092 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.419009    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq" event={"ID":"e15a33dd-590f-4670-bcad-c38e2d3db200","Type":"ContainerDied","Data":"0be95aa221472ac1b51e76375e334dc383d6c9a23f4e5c9e609c06bb27981db7"}
Jul 26 15:56:13.420496 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.420471    2559 generic.go:356] "Generic (PLEG): container finished" podID="d08107c5-1b5e-4f84-8bd7-1c6153aa360f" containerID="0d7368d6cb7fb3d91bd8e21cb5d8e715a516be095dcd659c9c2ddf021783f988" exitCode=0
Jul 26 15:56:13.420623 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.420502    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg"
Jul 26 15:56:13.420623 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.420506    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg" event={"ID":"d08107c5-1b5e-4f84-8bd7-1c6153aa360f","Type":"ContainerDied","Data":"0d7368d6cb7fb3d91bd8e21cb5d8e715a516be095dcd659c9c2ddf021783f988"}
Jul 26 15:56:13.420623 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.420545    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg" event={"ID":"d08107c5-1b5e-4f84-8bd7-1c6153aa360f","Type":"ContainerDied","Data":"9e4e0d811abe2651c66dfbd336fc59c3d3236da29b15854073bee92132639331"}
Jul 26 15:56:13.429098 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.429074    2559 scope.go:117] "RemoveContainer" containerID="342375ef3fade2d732b04b9e9793408b7093e2d08c66d59ebc73dc372e149e40"
Jul 26 15:56:13.438003 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.437982    2559 scope.go:117] "RemoveContainer" containerID="0d7368d6cb7fb3d91bd8e21cb5d8e715a516be095dcd659c9c2ddf021783f988"
Jul 26 15:56:13.449073 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.448111    2559 scope.go:117] "RemoveContainer" containerID="0d7368d6cb7fb3d91bd8e21cb5d8e715a516be095dcd659c9c2ddf021783f988"
Jul 26 15:56:13.449073 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.448686    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg"]
Jul 26 15:56:13.449073 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:56:13.448797    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d7368d6cb7fb3d91bd8e21cb5d8e715a516be095dcd659c9c2ddf021783f988\": container with ID starting with 0d7368d6cb7fb3d91bd8e21cb5d8e715a516be095dcd659c9c2ddf021783f988 not found: ID does not exist" containerID="0d7368d6cb7fb3d91bd8e21cb5d8e715a516be095dcd659c9c2ddf021783f988"
Jul 26 15:56:13.449073 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.448830    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d7368d6cb7fb3d91bd8e21cb5d8e715a516be095dcd659c9c2ddf021783f988"} err="failed to get container status \"0d7368d6cb7fb3d91bd8e21cb5d8e715a516be095dcd659c9c2ddf021783f988\": rpc error: code = NotFound desc = could not find container \"0d7368d6cb7fb3d91bd8e21cb5d8e715a516be095dcd659c9c2ddf021783f988\": container with ID starting with 0d7368d6cb7fb3d91bd8e21cb5d8e715a516be095dcd659c9c2ddf021783f988 not found: ID does not exist"
Jul 26 15:56:13.451297 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.451266    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-ff0984-74b8864b84-5hmzg"]
Jul 26 15:56:13.461032 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.461006    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"]
Jul 26 15:56:13.462787 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.462766    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ff0984-6bd94db6dc-cbknk"]
Jul 26 15:56:13.471993 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.471905    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq"]
Jul 26 15:56:13.475848 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:13.475819    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ff0984-84f4967d4d-2wcdq"]
Jul 26 15:56:14.195194 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:14.195156    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="052f9988-9747-47fe-9850-00fb4cf7a2a4" path="/var/lib/kubelet/pods/052f9988-9747-47fe-9850-00fb4cf7a2a4/volumes"
Jul 26 15:56:14.195582 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:14.195567    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d08107c5-1b5e-4f84-8bd7-1c6153aa360f" path="/var/lib/kubelet/pods/d08107c5-1b5e-4f84-8bd7-1c6153aa360f/volumes"
Jul 26 15:56:14.195912 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:14.195900    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e15a33dd-590f-4670-bcad-c38e2d3db200" path="/var/lib/kubelet/pods/e15a33dd-590f-4670-bcad-c38e2d3db200/volumes"
Jul 26 15:56:16.756418 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:16.756388    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:17.125235 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:17.125184    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:22.904066 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:22.904028    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:23.004029 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:23.003998    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:23.170648 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:23.170549    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:23.397062 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:23.397021    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:33.910171 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:33.910137    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"]
Jul 26 15:56:33.910801 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:33.910768    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq" podUID="dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6" containerName="maas-api" containerID="cri-o://f137dc163029d9d024cce31ab73b5944276a023460c0a9ff6151709c40d6074a" gracePeriod=30
Jul 26 15:56:33.943843 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:33.943805    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:33.979352 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:33.979319    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9"]
Jul 26 15:56:33.979570 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:33.979530    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9" podUID="b8c1b62f-a850-414f-8b29-3b7f27b21196" containerName="payload-pre-processing" containerID="cri-o://1656f2a997907a2d086d316a4592a5bd3b9c15c954e349b70fdd5edfc4172566" gracePeriod=30
Jul 26 15:56:34.038763 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.038727    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:34.138813 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.138780    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:34.259168 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.258868    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9"
Jul 26 15:56:34.349150 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.349121    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7d7tw\" (UniqueName: \"kubernetes.io/projected/b8c1b62f-a850-414f-8b29-3b7f27b21196-kube-api-access-7d7tw\") pod \"b8c1b62f-a850-414f-8b29-3b7f27b21196\" (UID: \"b8c1b62f-a850-414f-8b29-3b7f27b21196\") "
Jul 26 15:56:34.349333 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.349212    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8c1b62f-a850-414f-8b29-3b7f27b21196-plugins-config-volume\") pod \"b8c1b62f-a850-414f-8b29-3b7f27b21196\" (UID: \"b8c1b62f-a850-414f-8b29-3b7f27b21196\") "
Jul 26 15:56:34.349709 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.349683    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8c1b62f-a850-414f-8b29-3b7f27b21196-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8c1b62f-a850-414f-8b29-3b7f27b21196" (UID: "b8c1b62f-a850-414f-8b29-3b7f27b21196"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:56:34.352535 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.352508    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8c1b62f-a850-414f-8b29-3b7f27b21196-kube-api-access-7d7tw" (OuterVolumeSpecName: "kube-api-access-7d7tw") pod "b8c1b62f-a850-414f-8b29-3b7f27b21196" (UID: "b8c1b62f-a850-414f-8b29-3b7f27b21196"). InnerVolumeSpecName "kube-api-access-7d7tw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:56:34.378403 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.378377    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"
Jul 26 15:56:34.450395 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.450303    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-maas-api-tls\") pod \"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6\" (UID: \"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6\") "
Jul 26 15:56:34.450395 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.450383    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-openshift-service-ca\") pod \"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6\" (UID: \"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6\") "
Jul 26 15:56:34.450636 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.450479    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8974\" (UniqueName: \"kubernetes.io/projected/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-kube-api-access-h8974\") pod \"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6\" (UID: \"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6\") "
Jul 26 15:56:34.450735 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.450714    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6" (UID: "dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:56:34.450786 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.450776    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8c1b62f-a850-414f-8b29-3b7f27b21196-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:56:34.450828 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.450791    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-openshift-service-ca\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:56:34.450828 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.450805    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7d7tw\" (UniqueName: \"kubernetes.io/projected/b8c1b62f-a850-414f-8b29-3b7f27b21196-kube-api-access-7d7tw\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:56:34.452462 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.452439    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-kube-api-access-h8974" (OuterVolumeSpecName: "kube-api-access-h8974") pod "dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6" (UID: "dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6"). InnerVolumeSpecName "kube-api-access-h8974". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:56:34.452540 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.452473    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6" (UID: "dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:56:34.529631 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.529598    2559 generic.go:356] "Generic (PLEG): container finished" podID="b8c1b62f-a850-414f-8b29-3b7f27b21196" containerID="1656f2a997907a2d086d316a4592a5bd3b9c15c954e349b70fdd5edfc4172566" exitCode=0
Jul 26 15:56:34.529797 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.529641    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9"
Jul 26 15:56:34.529797 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.529676    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9" event={"ID":"b8c1b62f-a850-414f-8b29-3b7f27b21196","Type":"ContainerDied","Data":"1656f2a997907a2d086d316a4592a5bd3b9c15c954e349b70fdd5edfc4172566"}
Jul 26 15:56:34.529797 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.529722    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9" event={"ID":"b8c1b62f-a850-414f-8b29-3b7f27b21196","Type":"ContainerDied","Data":"047a450ae489570e5d9c4d3dfdba4bf70ea335c94c6d9819971ebccaf7a55a06"}
Jul 26 15:56:34.529797 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.529753    2559 scope.go:117] "RemoveContainer" containerID="1656f2a997907a2d086d316a4592a5bd3b9c15c954e349b70fdd5edfc4172566"
Jul 26 15:56:34.531090 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.531070    2559 generic.go:356] "Generic (PLEG): container finished" podID="dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6" containerID="f137dc163029d9d024cce31ab73b5944276a023460c0a9ff6151709c40d6074a" exitCode=0
Jul 26 15:56:34.531225 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.531117    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"
Jul 26 15:56:34.531225 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.531148    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq" event={"ID":"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6","Type":"ContainerDied","Data":"f137dc163029d9d024cce31ab73b5944276a023460c0a9ff6151709c40d6074a"}
Jul 26 15:56:34.531225 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.531180    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq" event={"ID":"dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6","Type":"ContainerDied","Data":"64eb2250abf599af94b44448626ab0d44ed8246b45cb3e3220c1ef1263347cd7"}
Jul 26 15:56:34.538367 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.538352    2559 scope.go:117] "RemoveContainer" containerID="1656f2a997907a2d086d316a4592a5bd3b9c15c954e349b70fdd5edfc4172566"
Jul 26 15:56:34.538657 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:56:34.538630    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1656f2a997907a2d086d316a4592a5bd3b9c15c954e349b70fdd5edfc4172566\": container with ID starting with 1656f2a997907a2d086d316a4592a5bd3b9c15c954e349b70fdd5edfc4172566 not found: ID does not exist" containerID="1656f2a997907a2d086d316a4592a5bd3b9c15c954e349b70fdd5edfc4172566"
Jul 26 15:56:34.538748 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.538667    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1656f2a997907a2d086d316a4592a5bd3b9c15c954e349b70fdd5edfc4172566"} err="failed to get container status \"1656f2a997907a2d086d316a4592a5bd3b9c15c954e349b70fdd5edfc4172566\": rpc error: code = NotFound desc = could not find container \"1656f2a997907a2d086d316a4592a5bd3b9c15c954e349b70fdd5edfc4172566\": container with ID starting with 1656f2a997907a2d086d316a4592a5bd3b9c15c954e349b70fdd5edfc4172566 not found: ID does not exist"
Jul 26 15:56:34.538748 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.538683    2559 scope.go:117] "RemoveContainer" containerID="f137dc163029d9d024cce31ab73b5944276a023460c0a9ff6151709c40d6074a"
Jul 26 15:56:34.547041 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.547026    2559 scope.go:117] "RemoveContainer" containerID="f137dc163029d9d024cce31ab73b5944276a023460c0a9ff6151709c40d6074a"
Jul 26 15:56:34.547308 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:56:34.547289    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f137dc163029d9d024cce31ab73b5944276a023460c0a9ff6151709c40d6074a\": container with ID starting with f137dc163029d9d024cce31ab73b5944276a023460c0a9ff6151709c40d6074a not found: ID does not exist" containerID="f137dc163029d9d024cce31ab73b5944276a023460c0a9ff6151709c40d6074a"
Jul 26 15:56:34.547380 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.547312    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f137dc163029d9d024cce31ab73b5944276a023460c0a9ff6151709c40d6074a"} err="failed to get container status \"f137dc163029d9d024cce31ab73b5944276a023460c0a9ff6151709c40d6074a\": rpc error: code = NotFound desc = could not find container \"f137dc163029d9d024cce31ab73b5944276a023460c0a9ff6151709c40d6074a\": container with ID starting with f137dc163029d9d024cce31ab73b5944276a023460c0a9ff6151709c40d6074a not found: ID does not exist"
Jul 26 15:56:34.552165 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.552139    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-maas-api-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:56:34.552278 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.552170    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8974\" (UniqueName: \"kubernetes.io/projected/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6-kube-api-access-h8974\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:56:34.553882 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.553863    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9"]
Jul 26 15:56:34.557180 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.557153    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-8c61fd-57b88d9d88-htsg9"]
Jul 26 15:56:34.565305 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.565284    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"]
Jul 26 15:56:34.568995 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:34.568973    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8c61fd-677478655c-mnntq"]
Jul 26 15:56:36.194046 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:36.194013    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8c1b62f-a850-414f-8b29-3b7f27b21196" path="/var/lib/kubelet/pods/b8c1b62f-a850-414f-8b29-3b7f27b21196/volumes"
Jul 26 15:56:36.194432 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:36.194416    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6" path="/var/lib/kubelet/pods/dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6/volumes"
Jul 26 15:56:38.580765 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:38.580723    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:38.951465 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:38.951380    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:44.697740 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:44.697697    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:44.798776 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:44.798741    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:44.972658 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:44.972573    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:45.206759 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:45.206723    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:45.559922 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:45.559885    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:45.867061 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:45.867027    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:45.975590 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:45.975554    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:48.459660 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.459623    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:48.558681 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.558640    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:48.610865 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.610833    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"]
Jul 26 15:56:48.611250 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.611236    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8c1b62f-a850-414f-8b29-3b7f27b21196" containerName="payload-pre-processing"
Jul 26 15:56:48.611250 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.611250    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8c1b62f-a850-414f-8b29-3b7f27b21196" containerName="payload-pre-processing"
Jul 26 15:56:48.611354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.611259    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6" containerName="maas-api"
Jul 26 15:56:48.611354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.611265    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6" containerName="maas-api"
Jul 26 15:56:48.611354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.611280    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d08107c5-1b5e-4f84-8bd7-1c6153aa360f" containerName="payload-pre-processing"
Jul 26 15:56:48.611354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.611287    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d08107c5-1b5e-4f84-8bd7-1c6153aa360f" containerName="payload-pre-processing"
Jul 26 15:56:48.611354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.611292    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e15a33dd-590f-4670-bcad-c38e2d3db200" containerName="payload-processing"
Jul 26 15:56:48.611354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.611298    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e15a33dd-590f-4670-bcad-c38e2d3db200" containerName="payload-processing"
Jul 26 15:56:48.611354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.611303    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="052f9988-9747-47fe-9850-00fb4cf7a2a4" containerName="maas-api"
Jul 26 15:56:48.611354 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.611308    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="052f9988-9747-47fe-9850-00fb4cf7a2a4" containerName="maas-api"
Jul 26 15:56:48.611634 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.611369    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8c1b62f-a850-414f-8b29-3b7f27b21196" containerName="payload-pre-processing"
Jul 26 15:56:48.611634 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.611378    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc9680cd-038a-43f8-b5fb-e11e8f4d4ed6" containerName="maas-api"
Jul 26 15:56:48.611634 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.611384    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e15a33dd-590f-4670-bcad-c38e2d3db200" containerName="payload-processing"
Jul 26 15:56:48.611634 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.611391    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="052f9988-9747-47fe-9850-00fb4cf7a2a4" containerName="maas-api"
Jul 26 15:56:48.611634 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.611398    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d08107c5-1b5e-4f84-8bd7-1c6153aa360f" containerName="payload-pre-processing"
Jul 26 15:56:48.614544 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.614525    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"
Jul 26 15:56:48.616152 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.616134    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-126bb0-serving-cert\""
Jul 26 15:56:48.624134 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.624107    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"]
Jul 26 15:56:48.701609 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.701576    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:48.787178 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.787084    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27af3b2c-0fcd-4974-a67d-055c17dd776c-maas-api-tls\") pod \"maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9\" (UID: \"27af3b2c-0fcd-4974-a67d-055c17dd776c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"
Jul 26 15:56:48.787178 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.787142    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2vjs\" (UniqueName: \"kubernetes.io/projected/27af3b2c-0fcd-4974-a67d-055c17dd776c-kube-api-access-g2vjs\") pod \"maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9\" (UID: \"27af3b2c-0fcd-4974-a67d-055c17dd776c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"
Jul 26 15:56:48.787407 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.787250    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27af3b2c-0fcd-4974-a67d-055c17dd776c-openshift-service-ca\") pod \"maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9\" (UID: \"27af3b2c-0fcd-4974-a67d-055c17dd776c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"
Jul 26 15:56:48.888115 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.888075    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2vjs\" (UniqueName: \"kubernetes.io/projected/27af3b2c-0fcd-4974-a67d-055c17dd776c-kube-api-access-g2vjs\") pod \"maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9\" (UID: \"27af3b2c-0fcd-4974-a67d-055c17dd776c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"
Jul 26 15:56:48.888309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.888137    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27af3b2c-0fcd-4974-a67d-055c17dd776c-openshift-service-ca\") pod \"maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9\" (UID: \"27af3b2c-0fcd-4974-a67d-055c17dd776c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"
Jul 26 15:56:48.888309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.888228    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27af3b2c-0fcd-4974-a67d-055c17dd776c-maas-api-tls\") pod \"maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9\" (UID: \"27af3b2c-0fcd-4974-a67d-055c17dd776c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"
Jul 26 15:56:48.888896 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.888873    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27af3b2c-0fcd-4974-a67d-055c17dd776c-openshift-service-ca\") pod \"maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9\" (UID: \"27af3b2c-0fcd-4974-a67d-055c17dd776c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"
Jul 26 15:56:48.890667 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.890644    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27af3b2c-0fcd-4974-a67d-055c17dd776c-maas-api-tls\") pod \"maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9\" (UID: \"27af3b2c-0fcd-4974-a67d-055c17dd776c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"
Jul 26 15:56:48.896149 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.896120    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2vjs\" (UniqueName: \"kubernetes.io/projected/27af3b2c-0fcd-4974-a67d-055c17dd776c-kube-api-access-g2vjs\") pod \"maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9\" (UID: \"27af3b2c-0fcd-4974-a67d-055c17dd776c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"
Jul 26 15:56:48.916016 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.915983    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:48.925325 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:48.925293    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"
Jul 26 15:56:49.061793 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.061758    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"]
Jul 26 15:56:49.064135 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:56:49.064109    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27af3b2c_0fcd_4974_a67d_055c17dd776c.slice/crio-afc0d7c24453418794f5ffea6302d31180073eb8f3c9750b9a9fdac0ea6b29e4 WatchSource:0}: Error finding container afc0d7c24453418794f5ffea6302d31180073eb8f3c9750b9a9fdac0ea6b29e4: Status 404 returned error can't find the container with id afc0d7c24453418794f5ffea6302d31180073eb8f3c9750b9a9fdac0ea6b29e4
Jul 26 15:56:49.065592 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.065574    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 15:56:49.356615 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.356505    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:49.485818 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.485784    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q"]
Jul 26 15:56:49.490850 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.490827    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q"
Jul 26 15:56:49.498969 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.498787    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-126bb0\""
Jul 26 15:56:49.498969 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.498814    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-126bb0-dockercfg-m4zmt\""
Jul 26 15:56:49.502417 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.502394    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q"]
Jul 26 15:56:49.595464 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.595434    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78d1f063-5512-442e-8ff6-7d620b6a593f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q\" (UID: \"78d1f063-5512-442e-8ff6-7d620b6a593f\") " pod="openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q"
Jul 26 15:56:49.595612 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.595468    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv2fx\" (UniqueName: \"kubernetes.io/projected/78d1f063-5512-442e-8ff6-7d620b6a593f-kube-api-access-kv2fx\") pod \"payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q\" (UID: \"78d1f063-5512-442e-8ff6-7d620b6a593f\") " pod="openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q"
Jul 26 15:56:49.605861 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.605828    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9" event={"ID":"27af3b2c-0fcd-4974-a67d-055c17dd776c","Type":"ContainerStarted","Data":"cc76d3783215e4ff120c50800e57b873b9fa9f0676f7fc369114228ca455b5eb"}
Jul 26 15:56:49.606019 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.605886    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"
Jul 26 15:56:49.606019 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.605902    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9" event={"ID":"27af3b2c-0fcd-4974-a67d-055c17dd776c","Type":"ContainerStarted","Data":"afc0d7c24453418794f5ffea6302d31180073eb8f3c9750b9a9fdac0ea6b29e4"}
Jul 26 15:56:49.620526 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.620422    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9" podStartSLOduration=1.161690582 podStartE2EDuration="1.620400406s" totalImagesPullingTime="458.709824ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:56:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:56:49.619729475 +0000 UTC m=+2293.909561605" watchObservedRunningTime="2026-07-26 15:56:49.620400406 +0000 UTC m=+2293.910232525"
Jul 26 15:56:49.696804 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.696756    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78d1f063-5512-442e-8ff6-7d620b6a593f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q\" (UID: \"78d1f063-5512-442e-8ff6-7d620b6a593f\") " pod="openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q"
Jul 26 15:56:49.696961 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.696810    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv2fx\" (UniqueName: \"kubernetes.io/projected/78d1f063-5512-442e-8ff6-7d620b6a593f-kube-api-access-kv2fx\") pod \"payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q\" (UID: \"78d1f063-5512-442e-8ff6-7d620b6a593f\") " pod="openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q"
Jul 26 15:56:49.697482 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.697462    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78d1f063-5512-442e-8ff6-7d620b6a593f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q\" (UID: \"78d1f063-5512-442e-8ff6-7d620b6a593f\") " pod="openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q"
Jul 26 15:56:49.703969 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.703941    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv2fx\" (UniqueName: \"kubernetes.io/projected/78d1f063-5512-442e-8ff6-7d620b6a593f-kube-api-access-kv2fx\") pod \"payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q\" (UID: \"78d1f063-5512-442e-8ff6-7d620b6a593f\") " pod="openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q"
Jul 26 15:56:49.802702 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.802659    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q"
Jul 26 15:56:49.938106 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:49.938069    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q"]
Jul 26 15:56:49.941380 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:56:49.941344    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78d1f063_5512_442e_8ff6_7d620b6a593f.slice/crio-73dc41dedcfcd138e06c6b63921c73e3a38561f8d511b5deb871af0b0a0a6a6b WatchSource:0}: Error finding container 73dc41dedcfcd138e06c6b63921c73e3a38561f8d511b5deb871af0b0a0a6a6b: Status 404 returned error can't find the container with id 73dc41dedcfcd138e06c6b63921c73e3a38561f8d511b5deb871af0b0a0a6a6b
Jul 26 15:56:50.553400 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:50.553357    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:50.612604 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:50.612568    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q" event={"ID":"78d1f063-5512-442e-8ff6-7d620b6a593f","Type":"ContainerStarted","Data":"93b96d7c99dbc236924c70cb08aae2ff60da60b2edd64616e79e045c338226f5"}
Jul 26 15:56:50.612780 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:50.612622    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q" event={"ID":"78d1f063-5512-442e-8ff6-7d620b6a593f","Type":"ContainerStarted","Data":"73dc41dedcfcd138e06c6b63921c73e3a38561f8d511b5deb871af0b0a0a6a6b"}
Jul 26 15:56:50.612829 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:50.612805    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q"
Jul 26 15:56:50.627979 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:50.627926    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q" podStartSLOduration=1.627910214 podStartE2EDuration="1.627910214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:56:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:56:50.626659852 +0000 UTC m=+2294.916491971" watchObservedRunningTime="2026-07-26 15:56:50.627910214 +0000 UTC m=+2294.917742334"
Jul 26 15:56:52.998221 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:52.998181    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:53.447259 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:53.447223    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:53.760154 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:53.760051    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:54.046499 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:54.046406    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:55.617687 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:55.617659    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"
Jul 26 15:56:56.523684 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:56.523651    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:56.611733 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:56.611690    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:56.612707 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:56.612687    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"]
Jul 26 15:56:56.616590 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:56.616570    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"
Jul 26 15:56:56.618664 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:56.618645    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-0da5f0-serving-cert\""
Jul 26 15:56:56.625410 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:56.625388    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"]
Jul 26 15:56:56.675347 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:56.675316    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:56.765754 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:56.765718    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b76b5ae6-6fa6-4adf-9d93-bd48592367ca-maas-api-tls\") pod \"maas-api-e2e-api-b-0da5f0-8695788474-lwbx2\" (UID: \"b76b5ae6-6fa6-4adf-9d93-bd48592367ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"
Jul 26 15:56:56.765946 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:56.765784    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b76b5ae6-6fa6-4adf-9d93-bd48592367ca-openshift-service-ca\") pod \"maas-api-e2e-api-b-0da5f0-8695788474-lwbx2\" (UID: \"b76b5ae6-6fa6-4adf-9d93-bd48592367ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"
Jul 26 15:56:56.765946 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:56.765917    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcrwx\" (UniqueName: \"kubernetes.io/projected/b76b5ae6-6fa6-4adf-9d93-bd48592367ca-kube-api-access-pcrwx\") pod \"maas-api-e2e-api-b-0da5f0-8695788474-lwbx2\" (UID: \"b76b5ae6-6fa6-4adf-9d93-bd48592367ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"
Jul 26 15:56:56.867142 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:56.867105    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b76b5ae6-6fa6-4adf-9d93-bd48592367ca-openshift-service-ca\") pod \"maas-api-e2e-api-b-0da5f0-8695788474-lwbx2\" (UID: \"b76b5ae6-6fa6-4adf-9d93-bd48592367ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"
Jul 26 15:56:56.867348 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:56.867179    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcrwx\" (UniqueName: \"kubernetes.io/projected/b76b5ae6-6fa6-4adf-9d93-bd48592367ca-kube-api-access-pcrwx\") pod \"maas-api-e2e-api-b-0da5f0-8695788474-lwbx2\" (UID: \"b76b5ae6-6fa6-4adf-9d93-bd48592367ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"
Jul 26 15:56:56.867348 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:56.867249    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b76b5ae6-6fa6-4adf-9d93-bd48592367ca-maas-api-tls\") pod \"maas-api-e2e-api-b-0da5f0-8695788474-lwbx2\" (UID: \"b76b5ae6-6fa6-4adf-9d93-bd48592367ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"
Jul 26 15:56:56.867741 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:56.867721    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b76b5ae6-6fa6-4adf-9d93-bd48592367ca-openshift-service-ca\") pod \"maas-api-e2e-api-b-0da5f0-8695788474-lwbx2\" (UID: \"b76b5ae6-6fa6-4adf-9d93-bd48592367ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"
Jul 26 15:56:56.869668 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:56.869645    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b76b5ae6-6fa6-4adf-9d93-bd48592367ca-maas-api-tls\") pod \"maas-api-e2e-api-b-0da5f0-8695788474-lwbx2\" (UID: \"b76b5ae6-6fa6-4adf-9d93-bd48592367ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"
Jul 26 15:56:56.874916 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:56.874891    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcrwx\" (UniqueName: \"kubernetes.io/projected/b76b5ae6-6fa6-4adf-9d93-bd48592367ca-kube-api-access-pcrwx\") pod \"maas-api-e2e-api-b-0da5f0-8695788474-lwbx2\" (UID: \"b76b5ae6-6fa6-4adf-9d93-bd48592367ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"
Jul 26 15:56:56.923027 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:56.922992    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:56.929907 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:56.929881    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"
Jul 26 15:56:57.071440 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.071351    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"]
Jul 26 15:56:57.073935 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:56:57.073905    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76b5ae6_6fa6_4adf_9d93_bd48592367ca.slice/crio-c24a4a3c9ac9ddd2002249d396c462b6844c2d62229e6b99f64ef43b153c8119 WatchSource:0}: Error finding container c24a4a3c9ac9ddd2002249d396c462b6844c2d62229e6b99f64ef43b153c8119: Status 404 returned error can't find the container with id c24a4a3c9ac9ddd2002249d396c462b6844c2d62229e6b99f64ef43b153c8119
Jul 26 15:56:57.218269 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.218160    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:57.305742 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.305709    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:56:57.354715 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.354678    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns"]
Jul 26 15:56:57.359483 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.359463    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns"
Jul 26 15:56:57.361349 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.361323    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-0da5f0\""
Jul 26 15:56:57.361479 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.361376    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-0da5f0-dockercfg-k4848\""
Jul 26 15:56:57.368479 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.368454    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns"]
Jul 26 15:56:57.372002 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.371978    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-877gl\" (UniqueName: \"kubernetes.io/projected/4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7-kube-api-access-877gl\") pod \"payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns\" (UID: \"4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7\") " pod="openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns"
Jul 26 15:56:57.372111 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.372080    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns\" (UID: \"4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7\") " pod="openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns"
Jul 26 15:56:57.472852 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.472736    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns\" (UID: \"4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7\") " pod="openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns"
Jul 26 15:56:57.473099 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.473015    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-877gl\" (UniqueName: \"kubernetes.io/projected/4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7-kube-api-access-877gl\") pod \"payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns\" (UID: \"4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7\") " pod="openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns"
Jul 26 15:56:57.473603 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.473577    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns\" (UID: \"4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7\") " pod="openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns"
Jul 26 15:56:57.484601 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.484575    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-877gl\" (UniqueName: \"kubernetes.io/projected/4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7-kube-api-access-877gl\") pod \"payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns\" (UID: \"4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7\") " pod="openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns"
Jul 26 15:56:57.650681 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.650639    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2" event={"ID":"b76b5ae6-6fa6-4adf-9d93-bd48592367ca","Type":"ContainerStarted","Data":"ac439ca132c13dc953e4c84c0d195e4794ac1393a144fc02c3a8232f94573892"}
Jul 26 15:56:57.651091 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.650704    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2" event={"ID":"b76b5ae6-6fa6-4adf-9d93-bd48592367ca","Type":"ContainerStarted","Data":"c24a4a3c9ac9ddd2002249d396c462b6844c2d62229e6b99f64ef43b153c8119"}
Jul 26 15:56:57.651091 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.650846    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"
Jul 26 15:56:57.671970 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.671939    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns"
Jul 26 15:56:57.689884 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.689815    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2" podStartSLOduration=1.328035939 podStartE2EDuration="1.68979415s" totalImagesPullingTime="361.758211ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:56:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:56:57.687693441 +0000 UTC m=+2301.977525560" watchObservedRunningTime="2026-07-26 15:56:57.68979415 +0000 UTC m=+2301.979626270"
Jul 26 15:56:57.818459 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:57.818421    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns"]
Jul 26 15:56:57.819882 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:56:57.819850    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dd2c614_ea9b_4cd6_a208_b9fb46eb73a7.slice/crio-e2da80d494562c9ae625e906701e5e0be2d422fdc1135c20c801cb8c5db6f262 WatchSource:0}: Error finding container e2da80d494562c9ae625e906701e5e0be2d422fdc1135c20c801cb8c5db6f262: Status 404 returned error can't find the container with id e2da80d494562c9ae625e906701e5e0be2d422fdc1135c20c801cb8c5db6f262
Jul 26 15:56:58.656918 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:58.656885    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns" event={"ID":"4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7","Type":"ContainerStarted","Data":"b6e19b7884340e390f7007209b072d6f4d7cbce54895b8d28c03eff68bad1654"}
Jul 26 15:56:58.657358 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:58.656941    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns" event={"ID":"4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7","Type":"ContainerStarted","Data":"e2da80d494562c9ae625e906701e5e0be2d422fdc1135c20c801cb8c5db6f262"}
Jul 26 15:56:58.657358 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:58.657104    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns"
Jul 26 15:56:58.673218 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:58.673156    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns" podStartSLOduration=1.673141721 podStartE2EDuration="1.673141721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:56:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:56:58.671008692 +0000 UTC m=+2302.960840812" watchObservedRunningTime="2026-07-26 15:56:58.673141721 +0000 UTC m=+2302.962973840"
Jul 26 15:56:59.904079 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:56:59.904041    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:57:01.029148 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:01.029113    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:57:01.620171 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:01.620139    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q"
Jul 26 15:57:03.662222 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:03.662180    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"
Jul 26 15:57:09.665388 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:09.665358    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns"
Jul 26 15:57:24.719748 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:24.719708    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"]
Jul 26 15:57:24.720326 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:24.720027    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9" podUID="27af3b2c-0fcd-4974-a67d-055c17dd776c" containerName="maas-api" containerID="cri-o://cc76d3783215e4ff120c50800e57b873b9fa9f0676f7fc369114228ca455b5eb" gracePeriod=30
Jul 26 15:57:24.745259 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:24.745225    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:57:24.778259 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:24.775788    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q"]
Jul 26 15:57:24.778259 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:24.776059    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q" podUID="78d1f063-5512-442e-8ff6-7d620b6a593f" containerName="payload-processing" containerID="cri-o://93b96d7c99dbc236924c70cb08aae2ff60da60b2edd64616e79e045c338226f5" gracePeriod=30
Jul 26 15:57:24.845632 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:24.845599    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:57:24.946746 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:24.946661    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:57:25.030529 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.030505    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"
Jul 26 15:57:25.062318 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.062293    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q"
Jul 26 15:57:25.155127 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.155090    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78d1f063-5512-442e-8ff6-7d620b6a593f-plugins-config-volume\") pod \"78d1f063-5512-442e-8ff6-7d620b6a593f\" (UID: \"78d1f063-5512-442e-8ff6-7d620b6a593f\") "
Jul 26 15:57:25.155127 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.155135    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27af3b2c-0fcd-4974-a67d-055c17dd776c-maas-api-tls\") pod \"27af3b2c-0fcd-4974-a67d-055c17dd776c\" (UID: \"27af3b2c-0fcd-4974-a67d-055c17dd776c\") "
Jul 26 15:57:25.155432 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.155175    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27af3b2c-0fcd-4974-a67d-055c17dd776c-openshift-service-ca\") pod \"27af3b2c-0fcd-4974-a67d-055c17dd776c\" (UID: \"27af3b2c-0fcd-4974-a67d-055c17dd776c\") "
Jul 26 15:57:25.155432 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.155248    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2vjs\" (UniqueName: \"kubernetes.io/projected/27af3b2c-0fcd-4974-a67d-055c17dd776c-kube-api-access-g2vjs\") pod \"27af3b2c-0fcd-4974-a67d-055c17dd776c\" (UID: \"27af3b2c-0fcd-4974-a67d-055c17dd776c\") "
Jul 26 15:57:25.155568 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.155482    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kv2fx\" (UniqueName: \"kubernetes.io/projected/78d1f063-5512-442e-8ff6-7d620b6a593f-kube-api-access-kv2fx\") pod \"78d1f063-5512-442e-8ff6-7d620b6a593f\" (UID: \"78d1f063-5512-442e-8ff6-7d620b6a593f\") "
Jul 26 15:57:25.155568 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.155541    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78d1f063-5512-442e-8ff6-7d620b6a593f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "78d1f063-5512-442e-8ff6-7d620b6a593f" (UID: "78d1f063-5512-442e-8ff6-7d620b6a593f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:57:25.155679 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.155603    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27af3b2c-0fcd-4974-a67d-055c17dd776c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "27af3b2c-0fcd-4974-a67d-055c17dd776c" (UID: "27af3b2c-0fcd-4974-a67d-055c17dd776c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:57:25.155863 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.155844    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27af3b2c-0fcd-4974-a67d-055c17dd776c-openshift-service-ca\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:57:25.155926 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.155868    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78d1f063-5512-442e-8ff6-7d620b6a593f-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:57:25.157361 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.157339    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27af3b2c-0fcd-4974-a67d-055c17dd776c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "27af3b2c-0fcd-4974-a67d-055c17dd776c" (UID: "27af3b2c-0fcd-4974-a67d-055c17dd776c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:57:25.157464 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.157374    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27af3b2c-0fcd-4974-a67d-055c17dd776c-kube-api-access-g2vjs" (OuterVolumeSpecName: "kube-api-access-g2vjs") pod "27af3b2c-0fcd-4974-a67d-055c17dd776c" (UID: "27af3b2c-0fcd-4974-a67d-055c17dd776c"). InnerVolumeSpecName "kube-api-access-g2vjs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:57:25.157464 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.157394    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78d1f063-5512-442e-8ff6-7d620b6a593f-kube-api-access-kv2fx" (OuterVolumeSpecName: "kube-api-access-kv2fx") pod "78d1f063-5512-442e-8ff6-7d620b6a593f" (UID: "78d1f063-5512-442e-8ff6-7d620b6a593f"). InnerVolumeSpecName "kube-api-access-kv2fx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:57:25.256980 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.256892    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27af3b2c-0fcd-4974-a67d-055c17dd776c-maas-api-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:57:25.256980 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.256925    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2vjs\" (UniqueName: \"kubernetes.io/projected/27af3b2c-0fcd-4974-a67d-055c17dd776c-kube-api-access-g2vjs\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:57:25.256980 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.256935    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kv2fx\" (UniqueName: \"kubernetes.io/projected/78d1f063-5512-442e-8ff6-7d620b6a593f-kube-api-access-kv2fx\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:57:25.736751 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.736715    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:57:25.804665 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.804628    2559 generic.go:356] "Generic (PLEG): container finished" podID="27af3b2c-0fcd-4974-a67d-055c17dd776c" containerID="cc76d3783215e4ff120c50800e57b873b9fa9f0676f7fc369114228ca455b5eb" exitCode=0
Jul 26 15:57:25.804853 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.804677    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"
Jul 26 15:57:25.804853 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.804716    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9" event={"ID":"27af3b2c-0fcd-4974-a67d-055c17dd776c","Type":"ContainerDied","Data":"cc76d3783215e4ff120c50800e57b873b9fa9f0676f7fc369114228ca455b5eb"}
Jul 26 15:57:25.804853 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.804770    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9" event={"ID":"27af3b2c-0fcd-4974-a67d-055c17dd776c","Type":"ContainerDied","Data":"afc0d7c24453418794f5ffea6302d31180073eb8f3c9750b9a9fdac0ea6b29e4"}
Jul 26 15:57:25.804853 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.804803    2559 scope.go:117] "RemoveContainer" containerID="cc76d3783215e4ff120c50800e57b873b9fa9f0676f7fc369114228ca455b5eb"
Jul 26 15:57:25.806273 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.806089    2559 generic.go:356] "Generic (PLEG): container finished" podID="78d1f063-5512-442e-8ff6-7d620b6a593f" containerID="93b96d7c99dbc236924c70cb08aae2ff60da60b2edd64616e79e045c338226f5" exitCode=0
Jul 26 15:57:25.806273 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.806140    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q"
Jul 26 15:57:25.806273 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.806143    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q" event={"ID":"78d1f063-5512-442e-8ff6-7d620b6a593f","Type":"ContainerDied","Data":"93b96d7c99dbc236924c70cb08aae2ff60da60b2edd64616e79e045c338226f5"}
Jul 26 15:57:25.806273 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.806188    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q" event={"ID":"78d1f063-5512-442e-8ff6-7d620b6a593f","Type":"ContainerDied","Data":"73dc41dedcfcd138e06c6b63921c73e3a38561f8d511b5deb871af0b0a0a6a6b"}
Jul 26 15:57:25.815172 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.815155    2559 scope.go:117] "RemoveContainer" containerID="cc76d3783215e4ff120c50800e57b873b9fa9f0676f7fc369114228ca455b5eb"
Jul 26 15:57:25.815553 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:57:25.815464    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc76d3783215e4ff120c50800e57b873b9fa9f0676f7fc369114228ca455b5eb\": container with ID starting with cc76d3783215e4ff120c50800e57b873b9fa9f0676f7fc369114228ca455b5eb not found: ID does not exist" containerID="cc76d3783215e4ff120c50800e57b873b9fa9f0676f7fc369114228ca455b5eb"
Jul 26 15:57:25.815553 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.815489    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc76d3783215e4ff120c50800e57b873b9fa9f0676f7fc369114228ca455b5eb"} err="failed to get container status \"cc76d3783215e4ff120c50800e57b873b9fa9f0676f7fc369114228ca455b5eb\": rpc error: code = NotFound desc = could not find container \"cc76d3783215e4ff120c50800e57b873b9fa9f0676f7fc369114228ca455b5eb\": container with ID starting with cc76d3783215e4ff120c50800e57b873b9fa9f0676f7fc369114228ca455b5eb not found: ID does not exist"
Jul 26 15:57:25.815553 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.815505    2559 scope.go:117] "RemoveContainer" containerID="93b96d7c99dbc236924c70cb08aae2ff60da60b2edd64616e79e045c338226f5"
Jul 26 15:57:25.826083 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.826061    2559 scope.go:117] "RemoveContainer" containerID="93b96d7c99dbc236924c70cb08aae2ff60da60b2edd64616e79e045c338226f5"
Jul 26 15:57:25.826226 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.826191    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"]
Jul 26 15:57:25.826396 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:57:25.826376    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93b96d7c99dbc236924c70cb08aae2ff60da60b2edd64616e79e045c338226f5\": container with ID starting with 93b96d7c99dbc236924c70cb08aae2ff60da60b2edd64616e79e045c338226f5 not found: ID does not exist" containerID="93b96d7c99dbc236924c70cb08aae2ff60da60b2edd64616e79e045c338226f5"
Jul 26 15:57:25.826447 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.826401    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93b96d7c99dbc236924c70cb08aae2ff60da60b2edd64616e79e045c338226f5"} err="failed to get container status \"93b96d7c99dbc236924c70cb08aae2ff60da60b2edd64616e79e045c338226f5\": rpc error: code = NotFound desc = could not find container \"93b96d7c99dbc236924c70cb08aae2ff60da60b2edd64616e79e045c338226f5\": container with ID starting with 93b96d7c99dbc236924c70cb08aae2ff60da60b2edd64616e79e045c338226f5 not found: ID does not exist"
Jul 26 15:57:25.829862 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.829842    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-126bb0-7b4c95ccfb-thss9"]
Jul 26 15:57:25.838428 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.838408    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q"]
Jul 26 15:57:25.841663 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:25.841640    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-126bb0-5884459d94-q9s8q"]
Jul 26 15:57:26.196285 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:26.196251    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27af3b2c-0fcd-4974-a67d-055c17dd776c" path="/var/lib/kubelet/pods/27af3b2c-0fcd-4974-a67d-055c17dd776c/volumes"
Jul 26 15:57:26.196628 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:26.196615    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78d1f063-5512-442e-8ff6-7d620b6a593f" path="/var/lib/kubelet/pods/78d1f063-5512-442e-8ff6-7d620b6a593f/volumes"
Jul 26 15:57:29.051268 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:29.051178    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:57:29.420895 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:29.420860    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:57:35.154229 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:35.154169    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:57:35.210668 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:35.210639    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:57:35.418600 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:35.418518    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:57:35.640945 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:35.640906    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:57:51.215954 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.215916    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"]
Jul 26 15:57:51.216490 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.216184    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2" podUID="b76b5ae6-6fa6-4adf-9d93-bd48592367ca" containerName="maas-api" containerID="cri-o://ac439ca132c13dc953e4c84c0d195e4794ac1393a144fc02c3a8232f94573892" gracePeriod=30
Jul 26 15:57:51.236763 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.236736    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:57:51.264846 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.264811    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns"]
Jul 26 15:57:51.265123 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.265067    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns" podUID="4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7" containerName="payload-processing" containerID="cri-o://b6e19b7884340e390f7007209b072d6f4d7cbce54895b8d28c03eff68bad1654" gracePeriod=30
Jul 26 15:57:51.336265 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.336229    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:57:51.437341 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.437307    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:57:51.517160 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.517135    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"
Jul 26 15:57:51.549109 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.549087    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns"
Jul 26 15:57:51.600703 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.600671    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b76b5ae6-6fa6-4adf-9d93-bd48592367ca-openshift-service-ca\") pod \"b76b5ae6-6fa6-4adf-9d93-bd48592367ca\" (UID: \"b76b5ae6-6fa6-4adf-9d93-bd48592367ca\") "
Jul 26 15:57:51.600908 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.600870    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7-plugins-config-volume\") pod \"4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7\" (UID: \"4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7\") "
Jul 26 15:57:51.600908 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.600902    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcrwx\" (UniqueName: \"kubernetes.io/projected/b76b5ae6-6fa6-4adf-9d93-bd48592367ca-kube-api-access-pcrwx\") pod \"b76b5ae6-6fa6-4adf-9d93-bd48592367ca\" (UID: \"b76b5ae6-6fa6-4adf-9d93-bd48592367ca\") "
Jul 26 15:57:51.601027 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.600917    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-877gl\" (UniqueName: \"kubernetes.io/projected/4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7-kube-api-access-877gl\") pod \"4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7\" (UID: \"4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7\") "
Jul 26 15:57:51.601027 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.600953    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b76b5ae6-6fa6-4adf-9d93-bd48592367ca-maas-api-tls\") pod \"b76b5ae6-6fa6-4adf-9d93-bd48592367ca\" (UID: \"b76b5ae6-6fa6-4adf-9d93-bd48592367ca\") "
Jul 26 15:57:51.601232 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.601179    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b76b5ae6-6fa6-4adf-9d93-bd48592367ca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b76b5ae6-6fa6-4adf-9d93-bd48592367ca" (UID: "b76b5ae6-6fa6-4adf-9d93-bd48592367ca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:57:51.601363 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.601297    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b76b5ae6-6fa6-4adf-9d93-bd48592367ca-openshift-service-ca\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:57:51.601363 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.601309    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7" (UID: "4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:57:51.603108 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.603086    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b76b5ae6-6fa6-4adf-9d93-bd48592367ca-kube-api-access-pcrwx" (OuterVolumeSpecName: "kube-api-access-pcrwx") pod "b76b5ae6-6fa6-4adf-9d93-bd48592367ca" (UID: "b76b5ae6-6fa6-4adf-9d93-bd48592367ca"). InnerVolumeSpecName "kube-api-access-pcrwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:57:51.603193 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.603130    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b76b5ae6-6fa6-4adf-9d93-bd48592367ca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b76b5ae6-6fa6-4adf-9d93-bd48592367ca" (UID: "b76b5ae6-6fa6-4adf-9d93-bd48592367ca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:57:51.603193 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.603134    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7-kube-api-access-877gl" (OuterVolumeSpecName: "kube-api-access-877gl") pod "4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7" (UID: "4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7"). InnerVolumeSpecName "kube-api-access-877gl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:57:51.701993 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.701956    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:57:51.701993 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.701989    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcrwx\" (UniqueName: \"kubernetes.io/projected/b76b5ae6-6fa6-4adf-9d93-bd48592367ca-kube-api-access-pcrwx\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:57:51.701993 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.701998    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-877gl\" (UniqueName: \"kubernetes.io/projected/4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7-kube-api-access-877gl\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:57:51.702249 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.702007    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b76b5ae6-6fa6-4adf-9d93-bd48592367ca-maas-api-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 15:57:51.941118 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.941084    2559 generic.go:356] "Generic (PLEG): container finished" podID="b76b5ae6-6fa6-4adf-9d93-bd48592367ca" containerID="ac439ca132c13dc953e4c84c0d195e4794ac1393a144fc02c3a8232f94573892" exitCode=0
Jul 26 15:57:51.941318 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.941124    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"
Jul 26 15:57:51.941318 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.941173    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2" event={"ID":"b76b5ae6-6fa6-4adf-9d93-bd48592367ca","Type":"ContainerDied","Data":"ac439ca132c13dc953e4c84c0d195e4794ac1393a144fc02c3a8232f94573892"}
Jul 26 15:57:51.941318 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.941245    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2" event={"ID":"b76b5ae6-6fa6-4adf-9d93-bd48592367ca","Type":"ContainerDied","Data":"c24a4a3c9ac9ddd2002249d396c462b6844c2d62229e6b99f64ef43b153c8119"}
Jul 26 15:57:51.941318 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.941274    2559 scope.go:117] "RemoveContainer" containerID="ac439ca132c13dc953e4c84c0d195e4794ac1393a144fc02c3a8232f94573892"
Jul 26 15:57:51.942463 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.942444    2559 generic.go:356] "Generic (PLEG): container finished" podID="4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7" containerID="b6e19b7884340e390f7007209b072d6f4d7cbce54895b8d28c03eff68bad1654" exitCode=0
Jul 26 15:57:51.942535 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.942474    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns" event={"ID":"4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7","Type":"ContainerDied","Data":"b6e19b7884340e390f7007209b072d6f4d7cbce54895b8d28c03eff68bad1654"}
Jul 26 15:57:51.942535 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.942495    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns"
Jul 26 15:57:51.942535 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.942518    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns" event={"ID":"4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7","Type":"ContainerDied","Data":"e2da80d494562c9ae625e906701e5e0be2d422fdc1135c20c801cb8c5db6f262"}
Jul 26 15:57:51.949705 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.949688    2559 scope.go:117] "RemoveContainer" containerID="ac439ca132c13dc953e4c84c0d195e4794ac1393a144fc02c3a8232f94573892"
Jul 26 15:57:51.949961 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:57:51.949938    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac439ca132c13dc953e4c84c0d195e4794ac1393a144fc02c3a8232f94573892\": container with ID starting with ac439ca132c13dc953e4c84c0d195e4794ac1393a144fc02c3a8232f94573892 not found: ID does not exist" containerID="ac439ca132c13dc953e4c84c0d195e4794ac1393a144fc02c3a8232f94573892"
Jul 26 15:57:51.950066 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.949967    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac439ca132c13dc953e4c84c0d195e4794ac1393a144fc02c3a8232f94573892"} err="failed to get container status \"ac439ca132c13dc953e4c84c0d195e4794ac1393a144fc02c3a8232f94573892\": rpc error: code = NotFound desc = could not find container \"ac439ca132c13dc953e4c84c0d195e4794ac1393a144fc02c3a8232f94573892\": container with ID starting with ac439ca132c13dc953e4c84c0d195e4794ac1393a144fc02c3a8232f94573892 not found: ID does not exist"
Jul 26 15:57:51.950066 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.949984    2559 scope.go:117] "RemoveContainer" containerID="b6e19b7884340e390f7007209b072d6f4d7cbce54895b8d28c03eff68bad1654"
Jul 26 15:57:51.958329 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.958314    2559 scope.go:117] "RemoveContainer" containerID="b6e19b7884340e390f7007209b072d6f4d7cbce54895b8d28c03eff68bad1654"
Jul 26 15:57:51.958570 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:57:51.958554    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6e19b7884340e390f7007209b072d6f4d7cbce54895b8d28c03eff68bad1654\": container with ID starting with b6e19b7884340e390f7007209b072d6f4d7cbce54895b8d28c03eff68bad1654 not found: ID does not exist" containerID="b6e19b7884340e390f7007209b072d6f4d7cbce54895b8d28c03eff68bad1654"
Jul 26 15:57:51.958615 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.958577    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6e19b7884340e390f7007209b072d6f4d7cbce54895b8d28c03eff68bad1654"} err="failed to get container status \"b6e19b7884340e390f7007209b072d6f4d7cbce54895b8d28c03eff68bad1654\": rpc error: code = NotFound desc = could not find container \"b6e19b7884340e390f7007209b072d6f4d7cbce54895b8d28c03eff68bad1654\": container with ID starting with b6e19b7884340e390f7007209b072d6f4d7cbce54895b8d28c03eff68bad1654 not found: ID does not exist"
Jul 26 15:57:51.965955 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.965933    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns"]
Jul 26 15:57:51.969403 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.969381    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0da5f0-5fb789d754-9rjns"]
Jul 26 15:57:51.978417 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.978394    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"]
Jul 26 15:57:51.982324 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:51.982297    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0da5f0-8695788474-lwbx2"]
Jul 26 15:57:52.194803 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:52.194719    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7" path="/var/lib/kubelet/pods/4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7/volumes"
Jul 26 15:57:52.195117 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:52.195104    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b76b5ae6-6fa6-4adf-9d93-bd48592367ca" path="/var/lib/kubelet/pods/b76b5ae6-6fa6-4adf-9d93-bd48592367ca/volumes"
Jul 26 15:57:55.840540 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:55.840497    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:57:56.218371 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:57:56.218283    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:01.933479 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:01.933444    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:02.031875 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:02.031832    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:02.194379 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:02.194298    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:02.447347 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:02.447262    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:02.776596 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:02.776508    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:03.096113 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:03.096083    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:03.570733 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:03.570700    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:06.012062 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.012024    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:06.103477 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.103434    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:06.103870 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.103836    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"]
Jul 26 15:58:06.104309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.104294    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78d1f063-5512-442e-8ff6-7d620b6a593f" containerName="payload-processing"
Jul 26 15:58:06.104309 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.104310    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="78d1f063-5512-442e-8ff6-7d620b6a593f" containerName="payload-processing"
Jul 26 15:58:06.104408 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.104319    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b76b5ae6-6fa6-4adf-9d93-bd48592367ca" containerName="maas-api"
Jul 26 15:58:06.104408 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.104325    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b76b5ae6-6fa6-4adf-9d93-bd48592367ca" containerName="maas-api"
Jul 26 15:58:06.104408 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.104337    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7" containerName="payload-processing"
Jul 26 15:58:06.104408 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.104342    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7" containerName="payload-processing"
Jul 26 15:58:06.104408 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.104350    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27af3b2c-0fcd-4974-a67d-055c17dd776c" containerName="maas-api"
Jul 26 15:58:06.104408 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.104355    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="27af3b2c-0fcd-4974-a67d-055c17dd776c" containerName="maas-api"
Jul 26 15:58:06.104587 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.104417    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="27af3b2c-0fcd-4974-a67d-055c17dd776c" containerName="maas-api"
Jul 26 15:58:06.104587 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.104424    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b76b5ae6-6fa6-4adf-9d93-bd48592367ca" containerName="maas-api"
Jul 26 15:58:06.104587 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.104429    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="78d1f063-5512-442e-8ff6-7d620b6a593f" containerName="payload-processing"
Jul 26 15:58:06.104587 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.104437    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dd2c614-ea9b-4cd6-a208-b9fb46eb73a7" containerName="payload-processing"
Jul 26 15:58:06.109058 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.109041    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"
Jul 26 15:58:06.111011 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.110991    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-d81393-serving-cert\""
Jul 26 15:58:06.119250 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.119228    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"]
Jul 26 15:58:06.163702 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.163673    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:06.248944 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.248903    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab134f70-a1a1-4f4d-bb02-21e08eb439b6-maas-api-tls\") pod \"maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b\" (UID: \"ab134f70-a1a1-4f4d-bb02-21e08eb439b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"
Jul 26 15:58:06.249123 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.249034    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab134f70-a1a1-4f4d-bb02-21e08eb439b6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b\" (UID: \"ab134f70-a1a1-4f4d-bb02-21e08eb439b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"
Jul 26 15:58:06.249123 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.249068    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkwl9\" (UniqueName: \"kubernetes.io/projected/ab134f70-a1a1-4f4d-bb02-21e08eb439b6-kube-api-access-xkwl9\") pod \"maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b\" (UID: \"ab134f70-a1a1-4f4d-bb02-21e08eb439b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"
Jul 26 15:58:06.263860 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.263780    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:06.350441 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.350388    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkwl9\" (UniqueName: \"kubernetes.io/projected/ab134f70-a1a1-4f4d-bb02-21e08eb439b6-kube-api-access-xkwl9\") pod \"maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b\" (UID: \"ab134f70-a1a1-4f4d-bb02-21e08eb439b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"
Jul 26 15:58:06.350669 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.350499    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab134f70-a1a1-4f4d-bb02-21e08eb439b6-maas-api-tls\") pod \"maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b\" (UID: \"ab134f70-a1a1-4f4d-bb02-21e08eb439b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"
Jul 26 15:58:06.351273 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.351183    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab134f70-a1a1-4f4d-bb02-21e08eb439b6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b\" (UID: \"ab134f70-a1a1-4f4d-bb02-21e08eb439b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"
Jul 26 15:58:06.351759 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.351736    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab134f70-a1a1-4f4d-bb02-21e08eb439b6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b\" (UID: \"ab134f70-a1a1-4f4d-bb02-21e08eb439b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"
Jul 26 15:58:06.353403 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.353382    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab134f70-a1a1-4f4d-bb02-21e08eb439b6-maas-api-tls\") pod \"maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b\" (UID: \"ab134f70-a1a1-4f4d-bb02-21e08eb439b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"
Jul 26 15:58:06.358622 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.358597    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkwl9\" (UniqueName: \"kubernetes.io/projected/ab134f70-a1a1-4f4d-bb02-21e08eb439b6-kube-api-access-xkwl9\") pod \"maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b\" (UID: \"ab134f70-a1a1-4f4d-bb02-21e08eb439b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"
Jul 26 15:58:06.420729 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.420691    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"
Jul 26 15:58:06.557043 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.557020    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"]
Jul 26 15:58:06.559743 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:58:06.559716    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab134f70_a1a1_4f4d_bb02_21e08eb439b6.slice/crio-10342cba64c6b73a1af1b18d10535381e9b14b3415a7446273d841a15cbf82c4 WatchSource:0}: Error finding container 10342cba64c6b73a1af1b18d10535381e9b14b3415a7446273d841a15cbf82c4: Status 404 returned error can't find the container with id 10342cba64c6b73a1af1b18d10535381e9b14b3415a7446273d841a15cbf82c4
Jul 26 15:58:06.683810 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.683775    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:06.779596 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.779566    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:06.787057 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.787030    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb"]
Jul 26 15:58:06.791774 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.791758    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb"
Jul 26 15:58:06.793789 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.793771    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-d81393-dockercfg-znh48\""
Jul 26 15:58:06.793902 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.793835    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-d81393\""
Jul 26 15:58:06.799948 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.799923    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb"]
Jul 26 15:58:06.957527 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.957502    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10017692-bc17-469f-ab16-ea63137a60b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb\" (UID: \"10017692-bc17-469f-ab16-ea63137a60b7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb"
Jul 26 15:58:06.957661 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:06.957600    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf2kp\" (UniqueName: \"kubernetes.io/projected/10017692-bc17-469f-ab16-ea63137a60b7-kube-api-access-wf2kp\") pod \"payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb\" (UID: \"10017692-bc17-469f-ab16-ea63137a60b7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb"
Jul 26 15:58:07.019655 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:07.019617    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b" event={"ID":"ab134f70-a1a1-4f4d-bb02-21e08eb439b6","Type":"ContainerStarted","Data":"18c8433607d9cab0c9343c798567fd027af6de9e4c76b38178412c9c74f111a2"}
Jul 26 15:58:07.020029 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:07.019674    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b" event={"ID":"ab134f70-a1a1-4f4d-bb02-21e08eb439b6","Type":"ContainerStarted","Data":"10342cba64c6b73a1af1b18d10535381e9b14b3415a7446273d841a15cbf82c4"}
Jul 26 15:58:07.020029 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:07.019707    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"
Jul 26 15:58:07.041595 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:07.041542    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b" podStartSLOduration=0.708900083 podStartE2EDuration="1.041525959s" totalImagesPullingTime="332.625876ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:58:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:58:07.039988064 +0000 UTC m=+2371.329820181" watchObservedRunningTime="2026-07-26 15:58:07.041525959 +0000 UTC m=+2371.331358076"
Jul 26 15:58:07.058443 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:07.058400    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10017692-bc17-469f-ab16-ea63137a60b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb\" (UID: \"10017692-bc17-469f-ab16-ea63137a60b7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb"
Jul 26 15:58:07.058606 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:07.058526    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf2kp\" (UniqueName: \"kubernetes.io/projected/10017692-bc17-469f-ab16-ea63137a60b7-kube-api-access-wf2kp\") pod \"payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb\" (UID: \"10017692-bc17-469f-ab16-ea63137a60b7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb"
Jul 26 15:58:07.059096 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:07.059077    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10017692-bc17-469f-ab16-ea63137a60b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb\" (UID: \"10017692-bc17-469f-ab16-ea63137a60b7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb"
Jul 26 15:58:07.066661 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:07.066630    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf2kp\" (UniqueName: \"kubernetes.io/projected/10017692-bc17-469f-ab16-ea63137a60b7-kube-api-access-wf2kp\") pod \"payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb\" (UID: \"10017692-bc17-469f-ab16-ea63137a60b7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb"
Jul 26 15:58:07.105010 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:07.104976    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb"
Jul 26 15:58:07.253176 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:07.253141    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb"]
Jul 26 15:58:07.256180 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:58:07.256149    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10017692_bc17_469f_ab16_ea63137a60b7.slice/crio-ccfb822e516eb2afddd6c6abbe6ba03e3199903a6de1fd26f29273027926b7ba WatchSource:0}: Error finding container ccfb822e516eb2afddd6c6abbe6ba03e3199903a6de1fd26f29273027926b7ba: Status 404 returned error can't find the container with id ccfb822e516eb2afddd6c6abbe6ba03e3199903a6de1fd26f29273027926b7ba
Jul 26 15:58:08.026040 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:08.026004    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb" event={"ID":"10017692-bc17-469f-ab16-ea63137a60b7","Type":"ContainerStarted","Data":"01bbcf347c4aa43d832d9a986bc50b23a501b59a1f2d562b3a4d05d955be2139"}
Jul 26 15:58:08.026502 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:08.026089    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb" event={"ID":"10017692-bc17-469f-ab16-ea63137a60b7","Type":"ContainerStarted","Data":"ccfb822e516eb2afddd6c6abbe6ba03e3199903a6de1fd26f29273027926b7ba"}
Jul 26 15:58:08.026502 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:08.026281    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb"
Jul 26 15:58:08.043126 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:08.043072    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb" podStartSLOduration=2.043054869 podStartE2EDuration="2.043054869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:58:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:58:08.041128391 +0000 UTC m=+2372.330960527" watchObservedRunningTime="2026-07-26 15:58:08.043054869 +0000 UTC m=+2372.332886987"
Jul 26 15:58:10.472157 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:10.472122    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:10.751115 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:10.751025    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:11.065114 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:11.065078    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:11.498659 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:11.498556    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:13.031327 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:13.031297    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"
Jul 26 15:58:13.941228 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:13.941166    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:14.039722 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.039685    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:14.156268 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.156229    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"]
Jul 26 15:58:14.159863 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.159843    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"
Jul 26 15:58:14.162324 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.162306    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-976e7c-serving-cert\""
Jul 26 15:58:14.170594 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.170570    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"]
Jul 26 15:58:14.200627 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.200541    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:14.225382 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.225350    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26450585-ef16-4808-b7a5-051056852cc7-openshift-service-ca\") pod \"maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r\" (UID: \"26450585-ef16-4808-b7a5-051056852cc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"
Jul 26 15:58:14.225382 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.225386    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26450585-ef16-4808-b7a5-051056852cc7-maas-api-tls\") pod \"maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r\" (UID: \"26450585-ef16-4808-b7a5-051056852cc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"
Jul 26 15:58:14.225579 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.225525    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4f8q\" (UniqueName: \"kubernetes.io/projected/26450585-ef16-4808-b7a5-051056852cc7-kube-api-access-v4f8q\") pod \"maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r\" (UID: \"26450585-ef16-4808-b7a5-051056852cc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"
Jul 26 15:58:14.325971 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.325933    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4f8q\" (UniqueName: \"kubernetes.io/projected/26450585-ef16-4808-b7a5-051056852cc7-kube-api-access-v4f8q\") pod \"maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r\" (UID: \"26450585-ef16-4808-b7a5-051056852cc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"
Jul 26 15:58:14.326164 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.325980    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26450585-ef16-4808-b7a5-051056852cc7-openshift-service-ca\") pod \"maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r\" (UID: \"26450585-ef16-4808-b7a5-051056852cc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"
Jul 26 15:58:14.326164 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.326003    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26450585-ef16-4808-b7a5-051056852cc7-maas-api-tls\") pod \"maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r\" (UID: \"26450585-ef16-4808-b7a5-051056852cc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"
Jul 26 15:58:14.326164 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:58:14.326148    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-serving-cert: secret "maas-api-e2e-shared-b-976e7c-serving-cert" not found
Jul 26 15:58:14.326346 ip-10-0-139-110 kubenswrapper[2559]: E0726 15:58:14.326241    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26450585-ef16-4808-b7a5-051056852cc7-maas-api-tls podName:26450585-ef16-4808-b7a5-051056852cc7 nodeName:}" failed. No retries permitted until 2026-07-26 15:58:14.826186875 +0000 UTC m=+2379.116018971 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/26450585-ef16-4808-b7a5-051056852cc7-maas-api-tls") pod "maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r" (UID: "26450585-ef16-4808-b7a5-051056852cc7") : secret "maas-api-e2e-shared-b-976e7c-serving-cert" not found
Jul 26 15:58:14.326649 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.326630    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26450585-ef16-4808-b7a5-051056852cc7-openshift-service-ca\") pod \"maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r\" (UID: \"26450585-ef16-4808-b7a5-051056852cc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"
Jul 26 15:58:14.340404 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.340376    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4f8q\" (UniqueName: \"kubernetes.io/projected/26450585-ef16-4808-b7a5-051056852cc7-kube-api-access-v4f8q\") pod \"maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r\" (UID: \"26450585-ef16-4808-b7a5-051056852cc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"
Jul 26 15:58:14.565906 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.565875    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:14.829432 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.829333    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26450585-ef16-4808-b7a5-051056852cc7-maas-api-tls\") pod \"maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r\" (UID: \"26450585-ef16-4808-b7a5-051056852cc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"
Jul 26 15:58:14.831884 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.831849    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26450585-ef16-4808-b7a5-051056852cc7-maas-api-tls\") pod \"maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r\" (UID: \"26450585-ef16-4808-b7a5-051056852cc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"
Jul 26 15:58:14.869080 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.869043    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:14.966013 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:14.965977    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:15.003621 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:15.003570    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526"]
Jul 26 15:58:15.007442 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:15.007424    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526"
Jul 26 15:58:15.009850 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:15.009826    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-976e7c-dockercfg-j54ll\""
Jul 26 15:58:15.010651 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:15.010634    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-976e7c\""
Jul 26 15:58:15.017247 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:15.017221    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526"]
Jul 26 15:58:15.030660 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:15.030627    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxbhs\" (UniqueName: \"kubernetes.io/projected/eea021bd-a122-463e-b5b0-eb3de4ba9adc-kube-api-access-sxbhs\") pod \"payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526\" (UID: \"eea021bd-a122-463e-b5b0-eb3de4ba9adc\") " pod="openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526"
Jul 26 15:58:15.030785 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:15.030680    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eea021bd-a122-463e-b5b0-eb3de4ba9adc-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526\" (UID: \"eea021bd-a122-463e-b5b0-eb3de4ba9adc\") " pod="openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526"
Jul 26 15:58:15.072461 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:15.072429    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"
Jul 26 15:58:15.132462 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:15.132050    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxbhs\" (UniqueName: \"kubernetes.io/projected/eea021bd-a122-463e-b5b0-eb3de4ba9adc-kube-api-access-sxbhs\") pod \"payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526\" (UID: \"eea021bd-a122-463e-b5b0-eb3de4ba9adc\") " pod="openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526"
Jul 26 15:58:15.132462 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:15.132102    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eea021bd-a122-463e-b5b0-eb3de4ba9adc-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526\" (UID: \"eea021bd-a122-463e-b5b0-eb3de4ba9adc\") " pod="openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526"
Jul 26 15:58:15.132962 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:15.132937    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eea021bd-a122-463e-b5b0-eb3de4ba9adc-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526\" (UID: \"eea021bd-a122-463e-b5b0-eb3de4ba9adc\") " pod="openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526"
Jul 26 15:58:15.143303 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:15.143250    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxbhs\" (UniqueName: \"kubernetes.io/projected/eea021bd-a122-463e-b5b0-eb3de4ba9adc-kube-api-access-sxbhs\") pod \"payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526\" (UID: \"eea021bd-a122-463e-b5b0-eb3de4ba9adc\") " pod="openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526"
Jul 26 15:58:15.215895 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:15.215756    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"]
Jul 26 15:58:15.218337 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:58:15.218294    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26450585_ef16_4808_b7a5_051056852cc7.slice/crio-a219bdd6babaeb66a2199f36a492bb6b8d1d16f7a55753e4ccec607acc90990e WatchSource:0}: Error finding container a219bdd6babaeb66a2199f36a492bb6b8d1d16f7a55753e4ccec607acc90990e: Status 404 returned error can't find the container with id a219bdd6babaeb66a2199f36a492bb6b8d1d16f7a55753e4ccec607acc90990e
Jul 26 15:58:15.320086 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:15.320052    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526"
Jul 26 15:58:15.477220 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:15.464480    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526"]
Jul 26 15:58:15.487523 ip-10-0-139-110 kubenswrapper[2559]: W0726 15:58:15.487483    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeea021bd_a122_463e_b5b0_eb3de4ba9adc.slice/crio-4029fa72e49b35f0d07f907e8f7673eaf2b1cd7021c7061b70374b96a52cda4c WatchSource:0}: Error finding container 4029fa72e49b35f0d07f907e8f7673eaf2b1cd7021c7061b70374b96a52cda4c: Status 404 returned error can't find the container with id 4029fa72e49b35f0d07f907e8f7673eaf2b1cd7021c7061b70374b96a52cda4c
Jul 26 15:58:16.068606 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:16.068570    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r" event={"ID":"26450585-ef16-4808-b7a5-051056852cc7","Type":"ContainerStarted","Data":"ddc76576b63bccc36d02782ac70200a7e178855e82abcda75057c0d0f5e9845a"}
Jul 26 15:58:16.068841 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:16.068758    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"
Jul 26 15:58:16.068841 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:16.068808    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r" event={"ID":"26450585-ef16-4808-b7a5-051056852cc7","Type":"ContainerStarted","Data":"a219bdd6babaeb66a2199f36a492bb6b8d1d16f7a55753e4ccec607acc90990e"}
Jul 26 15:58:16.070320 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:16.070295    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526" event={"ID":"eea021bd-a122-463e-b5b0-eb3de4ba9adc","Type":"ContainerStarted","Data":"099a70ce044defbab34f2a57c7f18ffd690a7c405366bbe61eac7c525b400eaa"}
Jul 26 15:58:16.070450 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:16.070340    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526" event={"ID":"eea021bd-a122-463e-b5b0-eb3de4ba9adc","Type":"ContainerStarted","Data":"4029fa72e49b35f0d07f907e8f7673eaf2b1cd7021c7061b70374b96a52cda4c"}
Jul 26 15:58:16.070450 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:16.070371    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526"
Jul 26 15:58:16.085139 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:16.085085    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r" podStartSLOduration=1.749009907 podStartE2EDuration="2.08506689s" totalImagesPullingTime="336.056983ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:58:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:58:16.083820183 +0000 UTC m=+2380.373652305" watchObservedRunningTime="2026-07-26 15:58:16.08506689 +0000 UTC m=+2380.374899010"
Jul 26 15:58:16.098474 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:16.098433    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526" podStartSLOduration=2.098419621 podStartE2EDuration="2.098419621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:58:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:58:16.097060347 +0000 UTC m=+2380.386892464" watchObservedRunningTime="2026-07-26 15:58:16.098419621 +0000 UTC m=+2380.388251739"
Jul 26 15:58:18.447796 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:18.447767    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:58:19.033937 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:19.033909    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb"
Jul 26 15:58:22.079656 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:22.079620    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"
Jul 26 15:58:27.076940 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:27.076906    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526"
Jul 26 15:58:31.503903 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:58:31.503866    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:59:00.524620 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:59:00.524532    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:59:17.719409 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:59:17.719371    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 15:59:26.544754 ip-10-0-139-110 kubenswrapper[2559]: I0726 15:59:26.544719    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:01:01.413483 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:01:01.413454    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:01:01.503958 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:01:01.503931    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:01:20.442031 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:01:20.441991    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:03:29.662072 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:03:29.661999    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:03:30.835379 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:03:30.835348    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:03:44.009056 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:03:44.009015    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:04:17.127962 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:04:17.127929    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:05:15.480981 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:05:15.480860    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:05:16.425768 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:05:16.425692    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:05:31.820742 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:05:31.820708    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:05:36.947581 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:05:36.947546    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:05:57.269683 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:05:57.269639    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:06:00.876436 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:06:00.876401    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:07:12.990547 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:12.990449    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:07:13.810548 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:13.810516    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:07:14.821443 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:14.821411    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7574-140629612538160"]
Jul 26 16:07:14.825104 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:14.825079    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629612538160"
Jul 26 16:07:14.826949 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:14.826924    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-jlg2w\""
Jul 26 16:07:14.827979 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:14.827954    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-140629612538160"]
Jul 26 16:07:14.887295 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:14.887256    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nq24\" (UniqueName: \"kubernetes.io/projected/89c8a474-f74d-4c69-9b2e-42eedd050a52-kube-api-access-2nq24\") pod \"test-curl-7574-140629612538160\" (UID: \"89c8a474-f74d-4c69-9b2e-42eedd050a52\") " pod="opendatahub/test-curl-7574-140629612538160"
Jul 26 16:07:14.987993 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:14.987959    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nq24\" (UniqueName: \"kubernetes.io/projected/89c8a474-f74d-4c69-9b2e-42eedd050a52-kube-api-access-2nq24\") pod \"test-curl-7574-140629612538160\" (UID: \"89c8a474-f74d-4c69-9b2e-42eedd050a52\") " pod="opendatahub/test-curl-7574-140629612538160"
Jul 26 16:07:14.995734 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:14.995702    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nq24\" (UniqueName: \"kubernetes.io/projected/89c8a474-f74d-4c69-9b2e-42eedd050a52-kube-api-access-2nq24\") pod \"test-curl-7574-140629612538160\" (UID: \"89c8a474-f74d-4c69-9b2e-42eedd050a52\") " pod="opendatahub/test-curl-7574-140629612538160"
Jul 26 16:07:15.136139 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:15.136047    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629612538160"
Jul 26 16:07:15.275216 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:15.275179    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-140629612538160"]
Jul 26 16:07:15.277582 ip-10-0-139-110 kubenswrapper[2559]: W0726 16:07:15.277551    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89c8a474_f74d_4c69_9b2e_42eedd050a52.slice/crio-ccf13c720169f28d0f1381b597cf612cb01b9ec0b68d71e4a2d2c272b7253fb6 WatchSource:0}: Error finding container ccf13c720169f28d0f1381b597cf612cb01b9ec0b68d71e4a2d2c272b7253fb6: Status 404 returned error can't find the container with id ccf13c720169f28d0f1381b597cf612cb01b9ec0b68d71e4a2d2c272b7253fb6
Jul 26 16:07:15.278888 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:15.278868    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 16:07:15.818438 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:15.818402    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-140629612538160" event={"ID":"89c8a474-f74d-4c69-9b2e-42eedd050a52","Type":"ContainerStarted","Data":"ccf13c720169f28d0f1381b597cf612cb01b9ec0b68d71e4a2d2c272b7253fb6"}
Jul 26 16:07:16.825081 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:16.825048    2559 generic.go:356] "Generic (PLEG): container finished" podID="89c8a474-f74d-4c69-9b2e-42eedd050a52" containerID="be2a2b3de576d1f89d2481936b3e626d611202cb43ff797ef41e32b378111fe6" exitCode=0
Jul 26 16:07:16.825505 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:16.825090    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-140629612538160" event={"ID":"89c8a474-f74d-4c69-9b2e-42eedd050a52","Type":"ContainerDied","Data":"be2a2b3de576d1f89d2481936b3e626d611202cb43ff797ef41e32b378111fe6"}
Jul 26 16:07:17.972095 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:17.972069    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629612538160"
Jul 26 16:07:17.990341 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:17.990312    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7574-140629612538160_89c8a474-f74d-4c69-9b2e-42eedd050a52/test-curl-7574-140629612538160/0.log"
Jul 26 16:07:18.013286 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.013258    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7574-140629612538160"]
Jul 26 16:07:18.015073 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.015049    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7574-140629612538160"]
Jul 26 16:07:18.018707 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.018690    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nq24\" (UniqueName: \"kubernetes.io/projected/89c8a474-f74d-4c69-9b2e-42eedd050a52-kube-api-access-2nq24\") pod \"89c8a474-f74d-4c69-9b2e-42eedd050a52\" (UID: \"89c8a474-f74d-4c69-9b2e-42eedd050a52\") "
Jul 26 16:07:18.020838 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.020805    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89c8a474-f74d-4c69-9b2e-42eedd050a52-kube-api-access-2nq24" (OuterVolumeSpecName: "kube-api-access-2nq24") pod "89c8a474-f74d-4c69-9b2e-42eedd050a52" (UID: "89c8a474-f74d-4c69-9b2e-42eedd050a52"). InnerVolumeSpecName "kube-api-access-2nq24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:07:18.119430 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.119351    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nq24\" (UniqueName: \"kubernetes.io/projected/89c8a474-f74d-4c69-9b2e-42eedd050a52-kube-api-access-2nq24\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:07:18.141182 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.141150    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7574-140629612622128"]
Jul 26 16:07:18.141619 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.141600    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89c8a474-f74d-4c69-9b2e-42eedd050a52" containerName="test-curl-7574-140629612538160"
Jul 26 16:07:18.141619 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.141618    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="89c8a474-f74d-4c69-9b2e-42eedd050a52" containerName="test-curl-7574-140629612538160"
Jul 26 16:07:18.141820 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.141683    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="89c8a474-f74d-4c69-9b2e-42eedd050a52" containerName="test-curl-7574-140629612538160"
Jul 26 16:07:18.144614 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.144592    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629612622128"
Jul 26 16:07:18.150293 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.150267    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-140629612622128"]
Jul 26 16:07:18.193835 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.193797    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89c8a474-f74d-4c69-9b2e-42eedd050a52" path="/var/lib/kubelet/pods/89c8a474-f74d-4c69-9b2e-42eedd050a52/volumes"
Jul 26 16:07:18.220262 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.220230    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzjnh\" (UniqueName: \"kubernetes.io/projected/aa10980b-567d-414a-8721-bf49693be9e2-kube-api-access-pzjnh\") pod \"test-curl-7574-140629612622128\" (UID: \"aa10980b-567d-414a-8721-bf49693be9e2\") " pod="opendatahub/test-curl-7574-140629612622128"
Jul 26 16:07:18.321714 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.321675    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzjnh\" (UniqueName: \"kubernetes.io/projected/aa10980b-567d-414a-8721-bf49693be9e2-kube-api-access-pzjnh\") pod \"test-curl-7574-140629612622128\" (UID: \"aa10980b-567d-414a-8721-bf49693be9e2\") " pod="opendatahub/test-curl-7574-140629612622128"
Jul 26 16:07:18.328434 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.328407    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzjnh\" (UniqueName: \"kubernetes.io/projected/aa10980b-567d-414a-8721-bf49693be9e2-kube-api-access-pzjnh\") pod \"test-curl-7574-140629612622128\" (UID: \"aa10980b-567d-414a-8721-bf49693be9e2\") " pod="opendatahub/test-curl-7574-140629612622128"
Jul 26 16:07:18.455094 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.455013    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629612622128"
Jul 26 16:07:18.598055 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.598024    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-140629612622128"]
Jul 26 16:07:18.600302 ip-10-0-139-110 kubenswrapper[2559]: W0726 16:07:18.600278    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa10980b_567d_414a_8721_bf49693be9e2.slice/crio-bb4264791f4532acc8936eecaf8e13b174569ac9ca657d559cc95349bb8c78f1 WatchSource:0}: Error finding container bb4264791f4532acc8936eecaf8e13b174569ac9ca657d559cc95349bb8c78f1: Status 404 returned error can't find the container with id bb4264791f4532acc8936eecaf8e13b174569ac9ca657d559cc95349bb8c78f1
Jul 26 16:07:18.836630 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.836601    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629612538160"
Jul 26 16:07:18.836818 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.836610    2559 scope.go:117] "RemoveContainer" containerID="be2a2b3de576d1f89d2481936b3e626d611202cb43ff797ef41e32b378111fe6"
Jul 26 16:07:18.838022 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:18.837951    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-140629612622128" event={"ID":"aa10980b-567d-414a-8721-bf49693be9e2","Type":"ContainerStarted","Data":"bb4264791f4532acc8936eecaf8e13b174569ac9ca657d559cc95349bb8c78f1"}
Jul 26 16:07:19.845506 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:19.845473    2559 generic.go:356] "Generic (PLEG): container finished" podID="aa10980b-567d-414a-8721-bf49693be9e2" containerID="51370a42a1919eee806f210d22ef37216cf658e97dda23c41db32ecaa6255a7c" exitCode=0
Jul 26 16:07:19.845880 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:19.845519    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-140629612622128" event={"ID":"aa10980b-567d-414a-8721-bf49693be9e2","Type":"ContainerDied","Data":"51370a42a1919eee806f210d22ef37216cf658e97dda23c41db32ecaa6255a7c"}
Jul 26 16:07:20.993174 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:20.993147    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629612622128"
Jul 26 16:07:21.011213 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.011172    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7574-140629612622128_aa10980b-567d-414a-8721-bf49693be9e2/test-curl-7574-140629612622128/0.log"
Jul 26 16:07:21.036709 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.036670    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7574-140629612622128"]
Jul 26 16:07:21.041843 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.041817    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7574-140629612622128"]
Jul 26 16:07:21.047653 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.047632    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzjnh\" (UniqueName: \"kubernetes.io/projected/aa10980b-567d-414a-8721-bf49693be9e2-kube-api-access-pzjnh\") pod \"aa10980b-567d-414a-8721-bf49693be9e2\" (UID: \"aa10980b-567d-414a-8721-bf49693be9e2\") "
Jul 26 16:07:21.049630 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.049607    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa10980b-567d-414a-8721-bf49693be9e2-kube-api-access-pzjnh" (OuterVolumeSpecName: "kube-api-access-pzjnh") pod "aa10980b-567d-414a-8721-bf49693be9e2" (UID: "aa10980b-567d-414a-8721-bf49693be9e2"). InnerVolumeSpecName "kube-api-access-pzjnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:07:21.149054 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.148972    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzjnh\" (UniqueName: \"kubernetes.io/projected/aa10980b-567d-414a-8721-bf49693be9e2-kube-api-access-pzjnh\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:07:21.210470 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.210435    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7574-140629617579904"]
Jul 26 16:07:21.210859 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.210846    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa10980b-567d-414a-8721-bf49693be9e2" containerName="test-curl-7574-140629612622128"
Jul 26 16:07:21.210903 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.210861    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa10980b-567d-414a-8721-bf49693be9e2" containerName="test-curl-7574-140629612622128"
Jul 26 16:07:21.210943 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.210938    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa10980b-567d-414a-8721-bf49693be9e2" containerName="test-curl-7574-140629612622128"
Jul 26 16:07:21.213958 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.213943    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629617579904"
Jul 26 16:07:21.219502 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.219470    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-140629617579904"]
Jul 26 16:07:21.249932 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.249899    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb92r\" (UniqueName: \"kubernetes.io/projected/03a2d2bf-3702-49d5-b564-9331e00bc3dd-kube-api-access-bb92r\") pod \"test-curl-7574-140629617579904\" (UID: \"03a2d2bf-3702-49d5-b564-9331e00bc3dd\") " pod="opendatahub/test-curl-7574-140629617579904"
Jul 26 16:07:21.351660 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.351628    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb92r\" (UniqueName: \"kubernetes.io/projected/03a2d2bf-3702-49d5-b564-9331e00bc3dd-kube-api-access-bb92r\") pod \"test-curl-7574-140629617579904\" (UID: \"03a2d2bf-3702-49d5-b564-9331e00bc3dd\") " pod="opendatahub/test-curl-7574-140629617579904"
Jul 26 16:07:21.359072 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.359043    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb92r\" (UniqueName: \"kubernetes.io/projected/03a2d2bf-3702-49d5-b564-9331e00bc3dd-kube-api-access-bb92r\") pod \"test-curl-7574-140629617579904\" (UID: \"03a2d2bf-3702-49d5-b564-9331e00bc3dd\") " pod="opendatahub/test-curl-7574-140629617579904"
Jul 26 16:07:21.524387 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.524297    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629617579904"
Jul 26 16:07:21.656942 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.656908    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-140629617579904"]
Jul 26 16:07:21.658874 ip-10-0-139-110 kubenswrapper[2559]: W0726 16:07:21.658838    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod03a2d2bf_3702_49d5_b564_9331e00bc3dd.slice/crio-86e1d3eaf9c75e38c2d9a64cc7d6a69e0036226b2c90e54bf2cb859207845c07 WatchSource:0}: Error finding container 86e1d3eaf9c75e38c2d9a64cc7d6a69e0036226b2c90e54bf2cb859207845c07: Status 404 returned error can't find the container with id 86e1d3eaf9c75e38c2d9a64cc7d6a69e0036226b2c90e54bf2cb859207845c07
Jul 26 16:07:21.857159 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.857131    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb4264791f4532acc8936eecaf8e13b174569ac9ca657d559cc95349bb8c78f1"
Jul 26 16:07:21.857159 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.857129    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629612622128"
Jul 26 16:07:21.858417 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:21.858391    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-140629617579904" event={"ID":"03a2d2bf-3702-49d5-b564-9331e00bc3dd","Type":"ContainerStarted","Data":"86e1d3eaf9c75e38c2d9a64cc7d6a69e0036226b2c90e54bf2cb859207845c07"}
Jul 26 16:07:21.867771 ip-10-0-139-110 kubenswrapper[2559]: E0726 16:07:21.867740    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7574-140629612622128\" is forbidden: User \"system:node:ip-10-0-139-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-110.ec2.internal' and this object" podUID="aa10980b-567d-414a-8721-bf49693be9e2" pod="opendatahub/test-curl-7574-140629612622128"
Jul 26 16:07:22.194466 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:22.194385    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa10980b-567d-414a-8721-bf49693be9e2" path="/var/lib/kubelet/pods/aa10980b-567d-414a-8721-bf49693be9e2/volumes"
Jul 26 16:07:22.865288 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:22.865253    2559 generic.go:356] "Generic (PLEG): container finished" podID="03a2d2bf-3702-49d5-b564-9331e00bc3dd" containerID="1a0933b69dacf0c099481c92864bad3c8151869333d9d17f4a0f521720d14e32" exitCode=0
Jul 26 16:07:22.865470 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:22.865338    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-140629617579904" event={"ID":"03a2d2bf-3702-49d5-b564-9331e00bc3dd","Type":"ContainerDied","Data":"1a0933b69dacf0c099481c92864bad3c8151869333d9d17f4a0f521720d14e32"}
Jul 26 16:07:24.011131 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.011106    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629617579904"
Jul 26 16:07:24.030627 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.030604    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7574-140629617579904_03a2d2bf-3702-49d5-b564-9331e00bc3dd/test-curl-7574-140629617579904/0.log"
Jul 26 16:07:24.058320 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.058292    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7574-140629617579904"]
Jul 26 16:07:24.061525 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.061499    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7574-140629617579904"]
Jul 26 16:07:24.078749 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.078727    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bb92r\" (UniqueName: \"kubernetes.io/projected/03a2d2bf-3702-49d5-b564-9331e00bc3dd-kube-api-access-bb92r\") pod \"03a2d2bf-3702-49d5-b564-9331e00bc3dd\" (UID: \"03a2d2bf-3702-49d5-b564-9331e00bc3dd\") "
Jul 26 16:07:24.080878 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.080847    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03a2d2bf-3702-49d5-b564-9331e00bc3dd-kube-api-access-bb92r" (OuterVolumeSpecName: "kube-api-access-bb92r") pod "03a2d2bf-3702-49d5-b564-9331e00bc3dd" (UID: "03a2d2bf-3702-49d5-b564-9331e00bc3dd"). InnerVolumeSpecName "kube-api-access-bb92r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:07:24.179547 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.179457    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb92r\" (UniqueName: \"kubernetes.io/projected/03a2d2bf-3702-49d5-b564-9331e00bc3dd-kube-api-access-bb92r\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:07:24.185018 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.184990    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7574-140629617579904"]
Jul 26 16:07:24.185434 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.185421    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03a2d2bf-3702-49d5-b564-9331e00bc3dd" containerName="test-curl-7574-140629617579904"
Jul 26 16:07:24.185489 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.185436    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="03a2d2bf-3702-49d5-b564-9331e00bc3dd" containerName="test-curl-7574-140629617579904"
Jul 26 16:07:24.185526 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.185515    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="03a2d2bf-3702-49d5-b564-9331e00bc3dd" containerName="test-curl-7574-140629617579904"
Jul 26 16:07:24.188730 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.188699    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629617579904"
Jul 26 16:07:24.198085 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.198051    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03a2d2bf-3702-49d5-b564-9331e00bc3dd" path="/var/lib/kubelet/pods/03a2d2bf-3702-49d5-b564-9331e00bc3dd/volumes"
Jul 26 16:07:24.198599 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.198580    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-140629617579904"]
Jul 26 16:07:24.280168 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.280133    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwl2g\" (UniqueName: \"kubernetes.io/projected/652e0668-e0c9-4072-a23f-b5aac7d44b62-kube-api-access-kwl2g\") pod \"test-curl-7574-140629617579904\" (UID: \"652e0668-e0c9-4072-a23f-b5aac7d44b62\") " pod="opendatahub/test-curl-7574-140629617579904"
Jul 26 16:07:24.381428 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.381374    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwl2g\" (UniqueName: \"kubernetes.io/projected/652e0668-e0c9-4072-a23f-b5aac7d44b62-kube-api-access-kwl2g\") pod \"test-curl-7574-140629617579904\" (UID: \"652e0668-e0c9-4072-a23f-b5aac7d44b62\") " pod="opendatahub/test-curl-7574-140629617579904"
Jul 26 16:07:24.388915 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.388886    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwl2g\" (UniqueName: \"kubernetes.io/projected/652e0668-e0c9-4072-a23f-b5aac7d44b62-kube-api-access-kwl2g\") pod \"test-curl-7574-140629617579904\" (UID: \"652e0668-e0c9-4072-a23f-b5aac7d44b62\") " pod="opendatahub/test-curl-7574-140629617579904"
Jul 26 16:07:24.501513 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.501429    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629617579904"
Jul 26 16:07:24.634553 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.634521    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-140629617579904"]
Jul 26 16:07:24.636925 ip-10-0-139-110 kubenswrapper[2559]: W0726 16:07:24.636883    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod652e0668_e0c9_4072_a23f_b5aac7d44b62.slice/crio-485fb0b205a3e2e3ae0c97bec5c23bf678b0bbbbf5f2ed997b7c93a405606d11 WatchSource:0}: Error finding container 485fb0b205a3e2e3ae0c97bec5c23bf678b0bbbbf5f2ed997b7c93a405606d11: Status 404 returned error can't find the container with id 485fb0b205a3e2e3ae0c97bec5c23bf678b0bbbbf5f2ed997b7c93a405606d11
Jul 26 16:07:24.878630 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.878593    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-140629617579904" event={"ID":"652e0668-e0c9-4072-a23f-b5aac7d44b62","Type":"ContainerStarted","Data":"485fb0b205a3e2e3ae0c97bec5c23bf678b0bbbbf5f2ed997b7c93a405606d11"}
Jul 26 16:07:24.879805 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.879785    2559 scope.go:117] "RemoveContainer" containerID="1a0933b69dacf0c099481c92864bad3c8151869333d9d17f4a0f521720d14e32"
Jul 26 16:07:24.879899 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:24.879808    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629617579904"
Jul 26 16:07:25.885949 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:25.885916    2559 generic.go:356] "Generic (PLEG): container finished" podID="652e0668-e0c9-4072-a23f-b5aac7d44b62" containerID="055935ffa01d8306154a5f2633c452b9fb202497ec97f590336550a0fe07c763" exitCode=0
Jul 26 16:07:25.886404 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:25.885950    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-140629617579904" event={"ID":"652e0668-e0c9-4072-a23f-b5aac7d44b62","Type":"ContainerDied","Data":"055935ffa01d8306154a5f2633c452b9fb202497ec97f590336550a0fe07c763"}
Jul 26 16:07:27.032217 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.032183    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629617579904"
Jul 26 16:07:27.050669 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.050644    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7574-140629617579904_652e0668-e0c9-4072-a23f-b5aac7d44b62/test-curl-7574-140629617579904/0.log"
Jul 26 16:07:27.074905 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.074872    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7574-140629617579904"]
Jul 26 16:07:27.080970 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.080944    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7574-140629617579904"]
Jul 26 16:07:27.107790 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.107765    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwl2g\" (UniqueName: \"kubernetes.io/projected/652e0668-e0c9-4072-a23f-b5aac7d44b62-kube-api-access-kwl2g\") pod \"652e0668-e0c9-4072-a23f-b5aac7d44b62\" (UID: \"652e0668-e0c9-4072-a23f-b5aac7d44b62\") "
Jul 26 16:07:27.109867 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.109840    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/652e0668-e0c9-4072-a23f-b5aac7d44b62-kube-api-access-kwl2g" (OuterVolumeSpecName: "kube-api-access-kwl2g") pod "652e0668-e0c9-4072-a23f-b5aac7d44b62" (UID: "652e0668-e0c9-4072-a23f-b5aac7d44b62"). InnerVolumeSpecName "kube-api-access-kwl2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:07:27.209758 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.209672    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwl2g\" (UniqueName: \"kubernetes.io/projected/652e0668-e0c9-4072-a23f-b5aac7d44b62-kube-api-access-kwl2g\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:07:27.211411 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.211389    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7574-140629612553824"]
Jul 26 16:07:27.211834 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.211820    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="652e0668-e0c9-4072-a23f-b5aac7d44b62" containerName="test-curl-7574-140629617579904"
Jul 26 16:07:27.211875 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.211836    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="652e0668-e0c9-4072-a23f-b5aac7d44b62" containerName="test-curl-7574-140629617579904"
Jul 26 16:07:27.211910 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.211904    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="652e0668-e0c9-4072-a23f-b5aac7d44b62" containerName="test-curl-7574-140629617579904"
Jul 26 16:07:27.214925 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.214905    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629612553824"
Jul 26 16:07:27.221385 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.221360    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-140629612553824"]
Jul 26 16:07:27.310685 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.310650    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkmp8\" (UniqueName: \"kubernetes.io/projected/0aa56533-9768-44ed-a9a2-17af06f6acf2-kube-api-access-bkmp8\") pod \"test-curl-7574-140629612553824\" (UID: \"0aa56533-9768-44ed-a9a2-17af06f6acf2\") " pod="opendatahub/test-curl-7574-140629612553824"
Jul 26 16:07:27.411658 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.411626    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkmp8\" (UniqueName: \"kubernetes.io/projected/0aa56533-9768-44ed-a9a2-17af06f6acf2-kube-api-access-bkmp8\") pod \"test-curl-7574-140629612553824\" (UID: \"0aa56533-9768-44ed-a9a2-17af06f6acf2\") " pod="opendatahub/test-curl-7574-140629612553824"
Jul 26 16:07:27.419122 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.419078    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkmp8\" (UniqueName: \"kubernetes.io/projected/0aa56533-9768-44ed-a9a2-17af06f6acf2-kube-api-access-bkmp8\") pod \"test-curl-7574-140629612553824\" (UID: \"0aa56533-9768-44ed-a9a2-17af06f6acf2\") " pod="opendatahub/test-curl-7574-140629612553824"
Jul 26 16:07:27.525473 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.525396    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629612553824"
Jul 26 16:07:27.662886 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.662827    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-140629612553824"]
Jul 26 16:07:27.665798 ip-10-0-139-110 kubenswrapper[2559]: W0726 16:07:27.665769    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0aa56533_9768_44ed_a9a2_17af06f6acf2.slice/crio-87ebb54b46c9576c4d9de8b0feda09b4bd346ef425faa948839872a6cce91539 WatchSource:0}: Error finding container 87ebb54b46c9576c4d9de8b0feda09b4bd346ef425faa948839872a6cce91539: Status 404 returned error can't find the container with id 87ebb54b46c9576c4d9de8b0feda09b4bd346ef425faa948839872a6cce91539
Jul 26 16:07:27.897596 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.897556    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-140629612553824" event={"ID":"0aa56533-9768-44ed-a9a2-17af06f6acf2","Type":"ContainerStarted","Data":"87ebb54b46c9576c4d9de8b0feda09b4bd346ef425faa948839872a6cce91539"}
Jul 26 16:07:27.898725 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.898704    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629617579904"
Jul 26 16:07:27.898876 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:27.898739    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="485fb0b205a3e2e3ae0c97bec5c23bf678b0bbbbf5f2ed997b7c93a405606d11"
Jul 26 16:07:27.908381 ip-10-0-139-110 kubenswrapper[2559]: E0726 16:07:27.908352    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7574-140629617579904\" is forbidden: User \"system:node:ip-10-0-139-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-110.ec2.internal' and this object" podUID="652e0668-e0c9-4072-a23f-b5aac7d44b62" pod="opendatahub/test-curl-7574-140629617579904"
Jul 26 16:07:28.194416 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:28.194337    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="652e0668-e0c9-4072-a23f-b5aac7d44b62" path="/var/lib/kubelet/pods/652e0668-e0c9-4072-a23f-b5aac7d44b62/volumes"
Jul 26 16:07:28.905054 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:28.905020    2559 generic.go:356] "Generic (PLEG): container finished" podID="0aa56533-9768-44ed-a9a2-17af06f6acf2" containerID="43be5bef5f85c8089a96d1eace3ff8863db90e31c5a58f85d40e8f156bc2fe86" exitCode=0
Jul 26 16:07:28.905245 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:28.905066    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-140629612553824" event={"ID":"0aa56533-9768-44ed-a9a2-17af06f6acf2","Type":"ContainerDied","Data":"43be5bef5f85c8089a96d1eace3ff8863db90e31c5a58f85d40e8f156bc2fe86"}
Jul 26 16:07:30.048713 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.048689    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629612553824"
Jul 26 16:07:30.068090 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.068062    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7574-140629612553824_0aa56533-9768-44ed-a9a2-17af06f6acf2/test-curl-7574-140629612553824/0.log"
Jul 26 16:07:30.092977 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.092941    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7574-140629612553824"]
Jul 26 16:07:30.094733 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.094713    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7574-140629612553824"]
Jul 26 16:07:30.137336 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.137303    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkmp8\" (UniqueName: \"kubernetes.io/projected/0aa56533-9768-44ed-a9a2-17af06f6acf2-kube-api-access-bkmp8\") pod \"0aa56533-9768-44ed-a9a2-17af06f6acf2\" (UID: \"0aa56533-9768-44ed-a9a2-17af06f6acf2\") "
Jul 26 16:07:30.139383 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.139355    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aa56533-9768-44ed-a9a2-17af06f6acf2-kube-api-access-bkmp8" (OuterVolumeSpecName: "kube-api-access-bkmp8") pod "0aa56533-9768-44ed-a9a2-17af06f6acf2" (UID: "0aa56533-9768-44ed-a9a2-17af06f6acf2"). InnerVolumeSpecName "kube-api-access-bkmp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:07:30.193623 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.193543    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0aa56533-9768-44ed-a9a2-17af06f6acf2" path="/var/lib/kubelet/pods/0aa56533-9768-44ed-a9a2-17af06f6acf2/volumes"
Jul 26 16:07:30.215110 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.215077    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7574-140629612553824"]
Jul 26 16:07:30.215634 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.215613    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aa56533-9768-44ed-a9a2-17af06f6acf2" containerName="test-curl-7574-140629612553824"
Jul 26 16:07:30.215634 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.215634    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aa56533-9768-44ed-a9a2-17af06f6acf2" containerName="test-curl-7574-140629612553824"
Jul 26 16:07:30.215796 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.215740    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aa56533-9768-44ed-a9a2-17af06f6acf2" containerName="test-curl-7574-140629612553824"
Jul 26 16:07:30.218956 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.218936    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629612553824"
Jul 26 16:07:30.223150 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.223121    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-140629612553824"]
Jul 26 16:07:30.238035 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.238006    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkmp8\" (UniqueName: \"kubernetes.io/projected/0aa56533-9768-44ed-a9a2-17af06f6acf2-kube-api-access-bkmp8\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:07:30.339364 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.339327    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvt8r\" (UniqueName: \"kubernetes.io/projected/65739d3f-9cb4-44c2-965a-03c4b5d2bda0-kube-api-access-xvt8r\") pod \"test-curl-7574-140629612553824\" (UID: \"65739d3f-9cb4-44c2-965a-03c4b5d2bda0\") " pod="opendatahub/test-curl-7574-140629612553824"
Jul 26 16:07:30.440448 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.440407    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvt8r\" (UniqueName: \"kubernetes.io/projected/65739d3f-9cb4-44c2-965a-03c4b5d2bda0-kube-api-access-xvt8r\") pod \"test-curl-7574-140629612553824\" (UID: \"65739d3f-9cb4-44c2-965a-03c4b5d2bda0\") " pod="opendatahub/test-curl-7574-140629612553824"
Jul 26 16:07:30.447434 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.447366    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvt8r\" (UniqueName: \"kubernetes.io/projected/65739d3f-9cb4-44c2-965a-03c4b5d2bda0-kube-api-access-xvt8r\") pod \"test-curl-7574-140629612553824\" (UID: \"65739d3f-9cb4-44c2-965a-03c4b5d2bda0\") " pod="opendatahub/test-curl-7574-140629612553824"
Jul 26 16:07:30.530715 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.530678    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629612553824"
Jul 26 16:07:30.664767 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.664730    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-140629612553824"]
Jul 26 16:07:30.667508 ip-10-0-139-110 kubenswrapper[2559]: W0726 16:07:30.667478    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65739d3f_9cb4_44c2_965a_03c4b5d2bda0.slice/crio-c1d1a6fbe73c9304dee38f0f8dea425c9bb98da7ead260b5d8fd3c28d85c60f4 WatchSource:0}: Error finding container c1d1a6fbe73c9304dee38f0f8dea425c9bb98da7ead260b5d8fd3c28d85c60f4: Status 404 returned error can't find the container with id c1d1a6fbe73c9304dee38f0f8dea425c9bb98da7ead260b5d8fd3c28d85c60f4
Jul 26 16:07:30.916063 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.916035    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629612553824"
Jul 26 16:07:30.916289 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.916077    2559 scope.go:117] "RemoveContainer" containerID="43be5bef5f85c8089a96d1eace3ff8863db90e31c5a58f85d40e8f156bc2fe86"
Jul 26 16:07:30.917427 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:30.917293    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-140629612553824" event={"ID":"65739d3f-9cb4-44c2-965a-03c4b5d2bda0","Type":"ContainerStarted","Data":"c1d1a6fbe73c9304dee38f0f8dea425c9bb98da7ead260b5d8fd3c28d85c60f4"}
Jul 26 16:07:31.928096 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:31.928063    2559 generic.go:356] "Generic (PLEG): container finished" podID="65739d3f-9cb4-44c2-965a-03c4b5d2bda0" containerID="bbcb1780a91f969df929c1037dc2fd865351d19e982cdd05cbce5cc0e2632d5b" exitCode=0
Jul 26 16:07:31.928492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:31.928112    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-140629612553824" event={"ID":"65739d3f-9cb4-44c2-965a-03c4b5d2bda0","Type":"ContainerDied","Data":"bbcb1780a91f969df929c1037dc2fd865351d19e982cdd05cbce5cc0e2632d5b"}
Jul 26 16:07:33.069685 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:33.069661    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629612553824"
Jul 26 16:07:33.088740 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:33.088714    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7574-140629612553824_65739d3f-9cb4-44c2-965a-03c4b5d2bda0/test-curl-7574-140629612553824/0.log"
Jul 26 16:07:33.109611 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:33.109582    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7574-140629612553824"]
Jul 26 16:07:33.113251 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:33.113197    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7574-140629612553824"]
Jul 26 16:07:33.165073 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:33.165045    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvt8r\" (UniqueName: \"kubernetes.io/projected/65739d3f-9cb4-44c2-965a-03c4b5d2bda0-kube-api-access-xvt8r\") pod \"65739d3f-9cb4-44c2-965a-03c4b5d2bda0\" (UID: \"65739d3f-9cb4-44c2-965a-03c4b5d2bda0\") "
Jul 26 16:07:33.167122 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:33.167089    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65739d3f-9cb4-44c2-965a-03c4b5d2bda0-kube-api-access-xvt8r" (OuterVolumeSpecName: "kube-api-access-xvt8r") pod "65739d3f-9cb4-44c2-965a-03c4b5d2bda0" (UID: "65739d3f-9cb4-44c2-965a-03c4b5d2bda0"). InnerVolumeSpecName "kube-api-access-xvt8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:07:33.266831 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:33.266604    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvt8r\" (UniqueName: \"kubernetes.io/projected/65739d3f-9cb4-44c2-965a-03c4b5d2bda0-kube-api-access-xvt8r\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:07:33.620406 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:33.620369    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:07:33.934798 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:33.934713    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:07:33.940340 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:33.940315    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140629612553824"
Jul 26 16:07:33.940487 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:33.940355    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c1d1a6fbe73c9304dee38f0f8dea425c9bb98da7ead260b5d8fd3c28d85c60f4"
Jul 26 16:07:33.949835 ip-10-0-139-110 kubenswrapper[2559]: E0726 16:07:33.949794    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7574-140629612553824\" is forbidden: User \"system:node:ip-10-0-139-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-110.ec2.internal' and this object" podUID="65739d3f-9cb4-44c2-965a-03c4b5d2bda0" pod="opendatahub/test-curl-7574-140629612553824"
Jul 26 16:07:34.160480 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:34.160441    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:07:34.194705 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:34.194629    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65739d3f-9cb4-44c2-965a-03c4b5d2bda0" path="/var/lib/kubelet/pods/65739d3f-9cb4-44c2-965a-03c4b5d2bda0/volumes"
Jul 26 16:07:36.628637 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:36.628585    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:07:36.726216 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:36.726173    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:07:37.513876 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.513838    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"]
Jul 26 16:07:37.514340 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.514316    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65739d3f-9cb4-44c2-965a-03c4b5d2bda0" containerName="test-curl-7574-140629612553824"
Jul 26 16:07:37.514340 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.514334    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="65739d3f-9cb4-44c2-965a-03c4b5d2bda0" containerName="test-curl-7574-140629612553824"
Jul 26 16:07:37.514515 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.514414    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="65739d3f-9cb4-44c2-965a-03c4b5d2bda0" containerName="test-curl-7574-140629612553824"
Jul 26 16:07:37.518857 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.518830    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"
Jul 26 16:07:37.520825 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.520806    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-9a599d-serving-cert\""
Jul 26 16:07:37.527315 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.527285    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"]
Jul 26 16:07:37.548494 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.548442    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:07:37.607063 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.607027    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1eaac55a-a153-4554-81ff-b721a20d006e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87\" (UID: \"1eaac55a-a153-4554-81ff-b721a20d006e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"
Jul 26 16:07:37.607278 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.607138    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs54k\" (UniqueName: \"kubernetes.io/projected/1eaac55a-a153-4554-81ff-b721a20d006e-kube-api-access-zs54k\") pod \"maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87\" (UID: \"1eaac55a-a153-4554-81ff-b721a20d006e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"
Jul 26 16:07:37.607357 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.607269    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1eaac55a-a153-4554-81ff-b721a20d006e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87\" (UID: \"1eaac55a-a153-4554-81ff-b721a20d006e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"
Jul 26 16:07:37.646695 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.646635    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:07:37.708683 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.708646    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1eaac55a-a153-4554-81ff-b721a20d006e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87\" (UID: \"1eaac55a-a153-4554-81ff-b721a20d006e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"
Jul 26 16:07:37.708894 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.708762    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs54k\" (UniqueName: \"kubernetes.io/projected/1eaac55a-a153-4554-81ff-b721a20d006e-kube-api-access-zs54k\") pod \"maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87\" (UID: \"1eaac55a-a153-4554-81ff-b721a20d006e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"
Jul 26 16:07:37.708894 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.708861    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1eaac55a-a153-4554-81ff-b721a20d006e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87\" (UID: \"1eaac55a-a153-4554-81ff-b721a20d006e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"
Jul 26 16:07:37.709504 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.709478    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1eaac55a-a153-4554-81ff-b721a20d006e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87\" (UID: \"1eaac55a-a153-4554-81ff-b721a20d006e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"
Jul 26 16:07:37.711435 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.711416    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1eaac55a-a153-4554-81ff-b721a20d006e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87\" (UID: \"1eaac55a-a153-4554-81ff-b721a20d006e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"
Jul 26 16:07:37.715536 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.715491    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs54k\" (UniqueName: \"kubernetes.io/projected/1eaac55a-a153-4554-81ff-b721a20d006e-kube-api-access-zs54k\") pod \"maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87\" (UID: \"1eaac55a-a153-4554-81ff-b721a20d006e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"
Jul 26 16:07:37.830715 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.830444    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"
Jul 26 16:07:37.972865 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:37.972835    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"]
Jul 26 16:07:37.975569 ip-10-0-139-110 kubenswrapper[2559]: W0726 16:07:37.975540    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eaac55a_a153_4554_81ff_b721a20d006e.slice/crio-066c01f05fb3626d7071c8bec548f0b6c35fc09641ae75734faa728478fdd3c5 WatchSource:0}: Error finding container 066c01f05fb3626d7071c8bec548f0b6c35fc09641ae75734faa728478fdd3c5: Status 404 returned error can't find the container with id 066c01f05fb3626d7071c8bec548f0b6c35fc09641ae75734faa728478fdd3c5
Jul 26 16:07:38.002364 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.002336    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:07:38.102257 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.102162    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:07:38.109488 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.109462    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs"]
Jul 26 16:07:38.114297 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.114274    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs"
Jul 26 16:07:38.116302 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.116278    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-9a599d\""
Jul 26 16:07:38.116413 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.116290    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-9a599d-dockercfg-tvm22\""
Jul 26 16:07:38.123987 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.123961    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs"]
Jul 26 16:07:38.142801 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.142761    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8"]
Jul 26 16:07:38.146300 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.146277    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8"
Jul 26 16:07:38.154070 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.154038    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8"]
Jul 26 16:07:38.212913 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.212875    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8f87181-6f0e-4cae-9040-d3c7065971af-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8\" (UID: \"d8f87181-6f0e-4cae-9040-d3c7065971af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8"
Jul 26 16:07:38.213179 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.213001    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f0209b3-90b9-46d4-b4c6-2d4a7eb87902-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs\" (UID: \"1f0209b3-90b9-46d4-b4c6-2d4a7eb87902\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs"
Jul 26 16:07:38.213264 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.213186    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjqm5\" (UniqueName: \"kubernetes.io/projected/1f0209b3-90b9-46d4-b4c6-2d4a7eb87902-kube-api-access-mjqm5\") pod \"payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs\" (UID: \"1f0209b3-90b9-46d4-b4c6-2d4a7eb87902\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs"
Jul 26 16:07:38.213327 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.213268    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b6jx\" (UniqueName: \"kubernetes.io/projected/d8f87181-6f0e-4cae-9040-d3c7065971af-kube-api-access-9b6jx\") pod \"payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8\" (UID: \"d8f87181-6f0e-4cae-9040-d3c7065971af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8"
Jul 26 16:07:38.314522 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.314482    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f0209b3-90b9-46d4-b4c6-2d4a7eb87902-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs\" (UID: \"1f0209b3-90b9-46d4-b4c6-2d4a7eb87902\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs"
Jul 26 16:07:38.314709 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.314543    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjqm5\" (UniqueName: \"kubernetes.io/projected/1f0209b3-90b9-46d4-b4c6-2d4a7eb87902-kube-api-access-mjqm5\") pod \"payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs\" (UID: \"1f0209b3-90b9-46d4-b4c6-2d4a7eb87902\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs"
Jul 26 16:07:38.314709 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.314565    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b6jx\" (UniqueName: \"kubernetes.io/projected/d8f87181-6f0e-4cae-9040-d3c7065971af-kube-api-access-9b6jx\") pod \"payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8\" (UID: \"d8f87181-6f0e-4cae-9040-d3c7065971af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8"
Jul 26 16:07:38.314709 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.314594    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8f87181-6f0e-4cae-9040-d3c7065971af-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8\" (UID: \"d8f87181-6f0e-4cae-9040-d3c7065971af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8"
Jul 26 16:07:38.315334 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.315237    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8f87181-6f0e-4cae-9040-d3c7065971af-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8\" (UID: \"d8f87181-6f0e-4cae-9040-d3c7065971af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8"
Jul 26 16:07:38.315334 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.315240    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f0209b3-90b9-46d4-b4c6-2d4a7eb87902-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs\" (UID: \"1f0209b3-90b9-46d4-b4c6-2d4a7eb87902\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs"
Jul 26 16:07:38.322613 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.322588    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjqm5\" (UniqueName: \"kubernetes.io/projected/1f0209b3-90b9-46d4-b4c6-2d4a7eb87902-kube-api-access-mjqm5\") pod \"payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs\" (UID: \"1f0209b3-90b9-46d4-b4c6-2d4a7eb87902\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs"
Jul 26 16:07:38.322721 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.322596    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b6jx\" (UniqueName: \"kubernetes.io/projected/d8f87181-6f0e-4cae-9040-d3c7065971af-kube-api-access-9b6jx\") pod \"payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8\" (UID: \"d8f87181-6f0e-4cae-9040-d3c7065971af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8"
Jul 26 16:07:38.426806 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.426781    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs"
Jul 26 16:07:38.457622 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.457590    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8"
Jul 26 16:07:38.583882 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.583838    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs"]
Jul 26 16:07:38.588425 ip-10-0-139-110 kubenswrapper[2559]: W0726 16:07:38.588382    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f0209b3_90b9_46d4_b4c6_2d4a7eb87902.slice/crio-4110047e5d7f863b9351b03c0c8d6d2297a6c1d65f2c17feb27017d9b4adf0ba WatchSource:0}: Error finding container 4110047e5d7f863b9351b03c0c8d6d2297a6c1d65f2c17feb27017d9b4adf0ba: Status 404 returned error can't find the container with id 4110047e5d7f863b9351b03c0c8d6d2297a6c1d65f2c17feb27017d9b4adf0ba
Jul 26 16:07:38.620980 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.620956    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8"]
Jul 26 16:07:38.624584 ip-10-0-139-110 kubenswrapper[2559]: W0726 16:07:38.624557    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8f87181_6f0e_4cae_9040_d3c7065971af.slice/crio-2581d8c6037f402d5e6c84f7bf425090d611c360a1e6df8d62564763f1d64cfd WatchSource:0}: Error finding container 2581d8c6037f402d5e6c84f7bf425090d611c360a1e6df8d62564763f1d64cfd: Status 404 returned error can't find the container with id 2581d8c6037f402d5e6c84f7bf425090d611c360a1e6df8d62564763f1d64cfd
Jul 26 16:07:38.981237 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.981126    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8" event={"ID":"d8f87181-6f0e-4cae-9040-d3c7065971af","Type":"ContainerStarted","Data":"6a9cc51233886d527e264bcaac112b6079d3bca4a3c9dcd449d091588aeecd80"}
Jul 26 16:07:38.981677 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.981250    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8" event={"ID":"d8f87181-6f0e-4cae-9040-d3c7065971af","Type":"ContainerStarted","Data":"2581d8c6037f402d5e6c84f7bf425090d611c360a1e6df8d62564763f1d64cfd"}
Jul 26 16:07:38.981677 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.981380    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8"
Jul 26 16:07:38.982666 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.982641    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs" event={"ID":"1f0209b3-90b9-46d4-b4c6-2d4a7eb87902","Type":"ContainerStarted","Data":"7a3d12ffeb2fa3adfb12e5b65ea1f5e5911dc9d2a47686767c8c407249d20c90"}
Jul 26 16:07:38.982773 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.982686    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs" event={"ID":"1f0209b3-90b9-46d4-b4c6-2d4a7eb87902","Type":"ContainerStarted","Data":"4110047e5d7f863b9351b03c0c8d6d2297a6c1d65f2c17feb27017d9b4adf0ba"}
Jul 26 16:07:38.982773 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.982712    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs"
Jul 26 16:07:38.984116 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.984095    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87" event={"ID":"1eaac55a-a153-4554-81ff-b721a20d006e","Type":"ContainerStarted","Data":"ad5aacc85d437f5e14f847fc4aa34dee026251aec0ea7d1197b73d7fde861734"}
Jul 26 16:07:38.984258 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.984137    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87" event={"ID":"1eaac55a-a153-4554-81ff-b721a20d006e","Type":"ContainerStarted","Data":"066c01f05fb3626d7071c8bec548f0b6c35fc09641ae75734faa728478fdd3c5"}
Jul 26 16:07:38.984317 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.984263    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"
Jul 26 16:07:38.997014 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:38.996974    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8" podStartSLOduration=0.996962622 podStartE2EDuration="996.962622ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:07:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:07:38.995049698 +0000 UTC m=+2943.284881817" watchObservedRunningTime="2026-07-26 16:07:38.996962622 +0000 UTC m=+2943.286794739"
Jul 26 16:07:39.011237 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:39.011178    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs" podStartSLOduration=1.011165137 podStartE2EDuration="1.011165137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:07:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:07:39.009408335 +0000 UTC m=+2943.299240453" watchObservedRunningTime="2026-07-26 16:07:39.011165137 +0000 UTC m=+2943.300997255"
Jul 26 16:07:39.026856 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:39.026808    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87" podStartSLOduration=1.621889439 podStartE2EDuration="2.026792502s" totalImagesPullingTime="404.903063ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:07:39.024759164 +0000 UTC m=+2943.314591286" watchObservedRunningTime="2026-07-26 16:07:39.026792502 +0000 UTC m=+2943.316624619"
Jul 26 16:07:41.140694 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:41.140665    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:07:44.996321 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:44.996291    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"
Jul 26 16:07:49.992462 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:49.992411    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs" podUID="1f0209b3-90b9-46d4-b4c6-2d4a7eb87902" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.78:9004: connect: connection refused"
Jul 26 16:07:49.992908 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:49.992756    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8"
Jul 26 16:07:59.991255 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:07:59.991155    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs"
Jul 26 16:08:02.351142 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:02.351107    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:08:02.671542 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:02.671434    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:08:03.469717 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:03.469683    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:08:05.842951 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:05.842915    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:08:05.944642 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:05.944607    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:08:07.550713 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:07.550677    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"]
Jul 26 16:08:07.554710 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:07.554685    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"
Jul 26 16:08:07.556508 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:07.556489    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-51789d-serving-cert\""
Jul 26 16:08:07.563505 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:07.563468    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"]
Jul 26 16:08:07.584122 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:07.584090    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:08:07.597529 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:07.597498    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-maas-api-tls\") pod \"maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc\" (UID: \"f62eab5b-d00e-4abf-8054-b4a0272fd8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"
Jul 26 16:08:07.597674 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:07.597620    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc\" (UID: \"f62eab5b-d00e-4abf-8054-b4a0272fd8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"
Jul 26 16:08:07.597718 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:07.597672    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh8r8\" (UniqueName: \"kubernetes.io/projected/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-kube-api-access-xh8r8\") pod \"maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc\" (UID: \"f62eab5b-d00e-4abf-8054-b4a0272fd8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"
Jul 26 16:08:07.698284 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:07.698250    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh8r8\" (UniqueName: \"kubernetes.io/projected/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-kube-api-access-xh8r8\") pod \"maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc\" (UID: \"f62eab5b-d00e-4abf-8054-b4a0272fd8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"
Jul 26 16:08:07.698502 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:07.698315    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-maas-api-tls\") pod \"maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc\" (UID: \"f62eab5b-d00e-4abf-8054-b4a0272fd8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"
Jul 26 16:08:07.698502 ip-10-0-139-110 kubenswrapper[2559]: E0726 16:08:07.698419    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-serving-cert: secret "maas-api-e2e-ipp-b-51789d-serving-cert" not found
Jul 26 16:08:07.698502 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:07.698419    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc\" (UID: \"f62eab5b-d00e-4abf-8054-b4a0272fd8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"
Jul 26 16:08:07.698502 ip-10-0-139-110 kubenswrapper[2559]: E0726 16:08:07.698498    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-maas-api-tls podName:f62eab5b-d00e-4abf-8054-b4a0272fd8c7 nodeName:}" failed. No retries permitted until 2026-07-26 16:08:08.198477479 +0000 UTC m=+2972.488309591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-maas-api-tls") pod "maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc" (UID: "f62eab5b-d00e-4abf-8054-b4a0272fd8c7") : secret "maas-api-e2e-ipp-b-51789d-serving-cert" not found
Jul 26 16:08:07.699034 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:07.699015    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc\" (UID: \"f62eab5b-d00e-4abf-8054-b4a0272fd8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"
Jul 26 16:08:07.705965 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:07.705929    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh8r8\" (UniqueName: \"kubernetes.io/projected/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-kube-api-access-xh8r8\") pod \"maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc\" (UID: \"f62eab5b-d00e-4abf-8054-b4a0272fd8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"
Jul 26 16:08:07.834097 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:07.834064    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:08:08.033177 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.033146    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:08:08.135724 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.135638    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:08:08.143608 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.143580    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66"]
Jul 26 16:08:08.147344 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.147328    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66"
Jul 26 16:08:08.149184 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.149157    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-51789d-dockercfg-hxmgx\""
Jul 26 16:08:08.149349 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.149334    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-51789d\""
Jul 26 16:08:08.156384 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.156357    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66"]
Jul 26 16:08:08.169352 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.169325    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl"]
Jul 26 16:08:08.172949 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.172930    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl"
Jul 26 16:08:08.181304 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.181277    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl"]
Jul 26 16:08:08.203445 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.203408    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jj9p\" (UniqueName: \"kubernetes.io/projected/10a45dbb-6a48-4633-8c59-ee9f6272151b-kube-api-access-2jj9p\") pod \"payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66\" (UID: \"10a45dbb-6a48-4633-8c59-ee9f6272151b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66"
Jul 26 16:08:08.203620 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.203515    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlsbn\" (UniqueName: \"kubernetes.io/projected/48c36a65-596e-4d6f-bd85-c60b581d9fcd-kube-api-access-mlsbn\") pod \"payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl\" (UID: \"48c36a65-596e-4d6f-bd85-c60b581d9fcd\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl"
Jul 26 16:08:08.203620 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.203579    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48c36a65-596e-4d6f-bd85-c60b581d9fcd-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl\" (UID: \"48c36a65-596e-4d6f-bd85-c60b581d9fcd\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl"
Jul 26 16:08:08.203707 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.203619    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10a45dbb-6a48-4633-8c59-ee9f6272151b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66\" (UID: \"10a45dbb-6a48-4633-8c59-ee9f6272151b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66"
Jul 26 16:08:08.203707 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.203656    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-maas-api-tls\") pod \"maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc\" (UID: \"f62eab5b-d00e-4abf-8054-b4a0272fd8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"
Jul 26 16:08:08.206101 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.206068    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-maas-api-tls\") pod \"maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc\" (UID: \"f62eab5b-d00e-4abf-8054-b4a0272fd8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"
Jul 26 16:08:08.305169 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.305132    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jj9p\" (UniqueName: \"kubernetes.io/projected/10a45dbb-6a48-4633-8c59-ee9f6272151b-kube-api-access-2jj9p\") pod \"payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66\" (UID: \"10a45dbb-6a48-4633-8c59-ee9f6272151b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66"
Jul 26 16:08:08.305380 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.305294    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlsbn\" (UniqueName: \"kubernetes.io/projected/48c36a65-596e-4d6f-bd85-c60b581d9fcd-kube-api-access-mlsbn\") pod \"payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl\" (UID: \"48c36a65-596e-4d6f-bd85-c60b581d9fcd\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl"
Jul 26 16:08:08.305380 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.305370    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48c36a65-596e-4d6f-bd85-c60b581d9fcd-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl\" (UID: \"48c36a65-596e-4d6f-bd85-c60b581d9fcd\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl"
Jul 26 16:08:08.305518 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.305416    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10a45dbb-6a48-4633-8c59-ee9f6272151b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66\" (UID: \"10a45dbb-6a48-4633-8c59-ee9f6272151b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66"
Jul 26 16:08:08.306130 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.306106    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10a45dbb-6a48-4633-8c59-ee9f6272151b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66\" (UID: \"10a45dbb-6a48-4633-8c59-ee9f6272151b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66"
Jul 26 16:08:08.306130 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.306118    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48c36a65-596e-4d6f-bd85-c60b581d9fcd-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl\" (UID: \"48c36a65-596e-4d6f-bd85-c60b581d9fcd\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl"
Jul 26 16:08:08.312560 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.312538    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jj9p\" (UniqueName: \"kubernetes.io/projected/10a45dbb-6a48-4633-8c59-ee9f6272151b-kube-api-access-2jj9p\") pod \"payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66\" (UID: \"10a45dbb-6a48-4633-8c59-ee9f6272151b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66"
Jul 26 16:08:08.312860 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.312841    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlsbn\" (UniqueName: \"kubernetes.io/projected/48c36a65-596e-4d6f-bd85-c60b581d9fcd-kube-api-access-mlsbn\") pod \"payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl\" (UID: \"48c36a65-596e-4d6f-bd85-c60b581d9fcd\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl"
Jul 26 16:08:08.459883 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.459790    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66"
Jul 26 16:08:08.468952 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.468930    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"
Jul 26 16:08:08.486253 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.486228    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl"
Jul 26 16:08:08.648078 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.648048    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"]
Jul 26 16:08:08.651284 ip-10-0-139-110 kubenswrapper[2559]: W0726 16:08:08.651196    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf62eab5b_d00e_4abf_8054_b4a0272fd8c7.slice/crio-24ef0ef98d56c0b3b41a0c65364397f838b0a14850605e59c100e2dcf01b50d2 WatchSource:0}: Error finding container 24ef0ef98d56c0b3b41a0c65364397f838b0a14850605e59c100e2dcf01b50d2: Status 404 returned error can't find the container with id 24ef0ef98d56c0b3b41a0c65364397f838b0a14850605e59c100e2dcf01b50d2
Jul 26 16:08:08.832284 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.832246    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66"]
Jul 26 16:08:08.834225 ip-10-0-139-110 kubenswrapper[2559]: W0726 16:08:08.834173    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10a45dbb_6a48_4633_8c59_ee9f6272151b.slice/crio-7ada835e22a54f72bfb75100fad07ca190ec83d187ebea276edbb968a41561d4 WatchSource:0}: Error finding container 7ada835e22a54f72bfb75100fad07ca190ec83d187ebea276edbb968a41561d4: Status 404 returned error can't find the container with id 7ada835e22a54f72bfb75100fad07ca190ec83d187ebea276edbb968a41561d4
Jul 26 16:08:08.884616 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:08.884592    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl"]
Jul 26 16:08:08.889664 ip-10-0-139-110 kubenswrapper[2559]: W0726 16:08:08.889636    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48c36a65_596e_4d6f_bd85_c60b581d9fcd.slice/crio-039a6d11ab68b286dee8aa918c84eec6621cb85420a592329e17ef5d7f19c766 WatchSource:0}: Error finding container 039a6d11ab68b286dee8aa918c84eec6621cb85420a592329e17ef5d7f19c766: Status 404 returned error can't find the container with id 039a6d11ab68b286dee8aa918c84eec6621cb85420a592329e17ef5d7f19c766
Jul 26 16:08:09.145853 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:09.145805    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl" event={"ID":"48c36a65-596e-4d6f-bd85-c60b581d9fcd","Type":"ContainerStarted","Data":"3fff67e6bca45dc2794dbb42cef87ec8716fb3f6a3976b718f69119ef4c6e6bb"}
Jul 26 16:08:09.146048 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:09.145899    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl" event={"ID":"48c36a65-596e-4d6f-bd85-c60b581d9fcd","Type":"ContainerStarted","Data":"039a6d11ab68b286dee8aa918c84eec6621cb85420a592329e17ef5d7f19c766"}
Jul 26 16:08:09.146129 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:09.146104    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl"
Jul 26 16:08:09.148586 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:09.148473    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc" event={"ID":"f62eab5b-d00e-4abf-8054-b4a0272fd8c7","Type":"ContainerStarted","Data":"65b82c423c9951f618e80f84250eee834af26df2d6b866445ca73c820ab6d74f"}
Jul 26 16:08:09.148586 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:09.148519    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc" event={"ID":"f62eab5b-d00e-4abf-8054-b4a0272fd8c7","Type":"ContainerStarted","Data":"24ef0ef98d56c0b3b41a0c65364397f838b0a14850605e59c100e2dcf01b50d2"}
Jul 26 16:08:09.148586 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:09.148548    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"
Jul 26 16:08:09.150216 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:09.150182    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66" event={"ID":"10a45dbb-6a48-4633-8c59-ee9f6272151b","Type":"ContainerStarted","Data":"ed452b7803fc4bd38939dfb1d48d34c95d6348e732be5f6f85f0bd0287fac23e"}
Jul 26 16:08:09.150392 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:09.150370    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66" event={"ID":"10a45dbb-6a48-4633-8c59-ee9f6272151b","Type":"ContainerStarted","Data":"7ada835e22a54f72bfb75100fad07ca190ec83d187ebea276edbb968a41561d4"}
Jul 26 16:08:09.150536 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:09.150513    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66"
Jul 26 16:08:09.162012 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:09.161952    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl" podStartSLOduration=1.161936056 podStartE2EDuration="1.161936056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:08:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:08:09.160074773 +0000 UTC m=+2973.449906890" watchObservedRunningTime="2026-07-26 16:08:09.161936056 +0000 UTC m=+2973.451768153"
Jul 26 16:08:09.175927 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:09.175871    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc" podStartSLOduration=1.794425226 podStartE2EDuration="2.175855373s" totalImagesPullingTime="381.430147ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:08:09.173852799 +0000 UTC m=+2973.463684922" watchObservedRunningTime="2026-07-26 16:08:09.175855373 +0000 UTC m=+2973.465687494"
Jul 26 16:08:09.188131 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:09.188074    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66" podStartSLOduration=1.188055434 podStartE2EDuration="1.188055434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:08:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:08:09.186402524 +0000 UTC m=+2973.476234643" watchObservedRunningTime="2026-07-26 16:08:09.188055434 +0000 UTC m=+2973.477887555"
Jul 26 16:08:10.412994 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:10.412962    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:08:15.161597 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:15.161569    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"
Jul 26 16:08:20.158411 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:20.158375    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl"
Jul 26 16:08:20.158996 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:20.158743    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66"
Jul 26 16:08:27.885074 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:27.885041    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wsp9v_afc1d831-651b-4136-8a8e-8440f9a966c3/payload-processing/0.log"
Jul 26 16:08:28.011074 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:28.011036    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl_48c36a65-596e-4d6f-bd85-c60b581d9fcd/payload-processing/0.log"
Jul 26 16:08:31.331323 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:08:31.331294    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:12.997950 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:12.997915    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8_d8f87181-6f0e-4cae-9040-d3c7065971af/payload-processing/0.log"
Jul 26 16:09:13.135439 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:13.135410    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wsp9v_afc1d831-651b-4136-8a8e-8440f9a966c3/payload-processing/0.log"
Jul 26 16:09:13.265630 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:13.265538    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl_48c36a65-596e-4d6f-bd85-c60b581d9fcd/payload-processing/0.log"
Jul 26 16:09:13.543427 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:13.543348    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:13.878086 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:13.878051    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:14.112833 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:14.112801    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:17.363683 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:17.363644    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:17.470616 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:17.470585    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:20.001934 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.001897    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"]
Jul 26 16:09:20.005734 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.005713    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"
Jul 26 16:09:20.007533 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.007506    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-36235c-serving-cert\""
Jul 26 16:09:20.013065 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.013043    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"]
Jul 26 16:09:20.016461 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.016419    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsfk2\" (UniqueName: \"kubernetes.io/projected/e1c6227d-739a-4f24-a633-ad3a3468bc19-kube-api-access-nsfk2\") pod \"maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj\" (UID: \"e1c6227d-739a-4f24-a633-ad3a3468bc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"
Jul 26 16:09:20.016574 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.016460    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1c6227d-739a-4f24-a633-ad3a3468bc19-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj\" (UID: \"e1c6227d-739a-4f24-a633-ad3a3468bc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"
Jul 26 16:09:20.016646 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.016615    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1c6227d-739a-4f24-a633-ad3a3468bc19-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj\" (UID: \"e1c6227d-739a-4f24-a633-ad3a3468bc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"
Jul 26 16:09:20.028134 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.028109    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:20.117782 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.117738    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsfk2\" (UniqueName: \"kubernetes.io/projected/e1c6227d-739a-4f24-a633-ad3a3468bc19-kube-api-access-nsfk2\") pod \"maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj\" (UID: \"e1c6227d-739a-4f24-a633-ad3a3468bc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"
Jul 26 16:09:20.117782 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.117785    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1c6227d-739a-4f24-a633-ad3a3468bc19-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj\" (UID: \"e1c6227d-739a-4f24-a633-ad3a3468bc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"
Jul 26 16:09:20.118054 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.117894    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1c6227d-739a-4f24-a633-ad3a3468bc19-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj\" (UID: \"e1c6227d-739a-4f24-a633-ad3a3468bc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"
Jul 26 16:09:20.118054 ip-10-0-139-110 kubenswrapper[2559]: E0726 16:09:20.117927    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-serving-cert: secret "maas-api-e2e-ipp-cleanup-36235c-serving-cert" not found
Jul 26 16:09:20.118054 ip-10-0-139-110 kubenswrapper[2559]: E0726 16:09:20.117999    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1c6227d-739a-4f24-a633-ad3a3468bc19-maas-api-tls podName:e1c6227d-739a-4f24-a633-ad3a3468bc19 nodeName:}" failed. No retries permitted until 2026-07-26 16:09:20.6179844 +0000 UTC m=+3044.907816497 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e1c6227d-739a-4f24-a633-ad3a3468bc19-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj" (UID: "e1c6227d-739a-4f24-a633-ad3a3468bc19") : secret "maas-api-e2e-ipp-cleanup-36235c-serving-cert" not found
Jul 26 16:09:20.118581 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.118563    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1c6227d-739a-4f24-a633-ad3a3468bc19-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj\" (UID: \"e1c6227d-739a-4f24-a633-ad3a3468bc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"
Jul 26 16:09:20.127427 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.127403    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsfk2\" (UniqueName: \"kubernetes.io/projected/e1c6227d-739a-4f24-a633-ad3a3468bc19-kube-api-access-nsfk2\") pod \"maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj\" (UID: \"e1c6227d-739a-4f24-a633-ad3a3468bc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"
Jul 26 16:09:20.262808 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.262720    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:20.466407 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.466374    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:20.564177 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.564143    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:20.644731 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.644702    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1c6227d-739a-4f24-a633-ad3a3468bc19-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj\" (UID: \"e1c6227d-739a-4f24-a633-ad3a3468bc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"
Jul 26 16:09:20.648807 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.648778    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1c6227d-739a-4f24-a633-ad3a3468bc19-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj\" (UID: \"e1c6227d-739a-4f24-a633-ad3a3468bc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"
Jul 26 16:09:20.920340 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:20.920252    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"
Jul 26 16:09:21.059979 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:21.059953    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"]
Jul 26 16:09:21.061862 ip-10-0-139-110 kubenswrapper[2559]: W0726 16:09:21.061820    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c6227d_739a_4f24_a633_ad3a3468bc19.slice/crio-89e548e149a6a34d7c73c68c71bd007128b91593c805f3f14a733b0de1e43cb1 WatchSource:0}: Error finding container 89e548e149a6a34d7c73c68c71bd007128b91593c805f3f14a733b0de1e43cb1: Status 404 returned error can't find the container with id 89e548e149a6a34d7c73c68c71bd007128b91593c805f3f14a733b0de1e43cb1
Jul 26 16:09:21.563271 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:21.563240    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj" event={"ID":"e1c6227d-739a-4f24-a633-ad3a3468bc19","Type":"ContainerStarted","Data":"89e548e149a6a34d7c73c68c71bd007128b91593c805f3f14a733b0de1e43cb1"}
Jul 26 16:09:21.823494 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:21.823450    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:22.569899 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:22.569860    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj" event={"ID":"e1c6227d-739a-4f24-a633-ad3a3468bc19","Type":"ContainerStarted","Data":"34168c013c611409bce356ca737af1c0228b6fb9edcda9b936167082a0030855"}
Jul 26 16:09:22.570306 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:22.569987    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"
Jul 26 16:09:22.586516 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:22.586471    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj" podStartSLOduration=3.119959859 podStartE2EDuration="3.586454953s" totalImagesPullingTime="466.495094ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:09:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:22.584760012 +0000 UTC m=+3046.874592123" watchObservedRunningTime="2026-07-26 16:09:22.586454953 +0000 UTC m=+3046.876287078"
Jul 26 16:09:28.581731 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:28.581655    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"
Jul 26 16:09:48.923634 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:48.923597    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"]
Jul 26 16:09:48.924463 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:48.924404    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj" podUID="e1c6227d-739a-4f24-a633-ad3a3468bc19" containerName="maas-api" containerID="cri-o://34168c013c611409bce356ca737af1c0228b6fb9edcda9b936167082a0030855" gracePeriod=30
Jul 26 16:09:48.948185 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:48.948107    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:49.013428 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.013401    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:49.066178 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.066142    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:49.182887 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.182824    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"
Jul 26 16:09:49.230754 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.230709    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1c6227d-739a-4f24-a633-ad3a3468bc19-openshift-service-ca\") pod \"e1c6227d-739a-4f24-a633-ad3a3468bc19\" (UID: \"e1c6227d-739a-4f24-a633-ad3a3468bc19\") "
Jul 26 16:09:49.230942 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.230829    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsfk2\" (UniqueName: \"kubernetes.io/projected/e1c6227d-739a-4f24-a633-ad3a3468bc19-kube-api-access-nsfk2\") pod \"e1c6227d-739a-4f24-a633-ad3a3468bc19\" (UID: \"e1c6227d-739a-4f24-a633-ad3a3468bc19\") "
Jul 26 16:09:49.230942 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.230856    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1c6227d-739a-4f24-a633-ad3a3468bc19-maas-api-tls\") pod \"e1c6227d-739a-4f24-a633-ad3a3468bc19\" (UID: \"e1c6227d-739a-4f24-a633-ad3a3468bc19\") "
Jul 26 16:09:49.231216 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.231174    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1c6227d-739a-4f24-a633-ad3a3468bc19-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e1c6227d-739a-4f24-a633-ad3a3468bc19" (UID: "e1c6227d-739a-4f24-a633-ad3a3468bc19"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:09:49.232876 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.232855    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1c6227d-739a-4f24-a633-ad3a3468bc19-kube-api-access-nsfk2" (OuterVolumeSpecName: "kube-api-access-nsfk2") pod "e1c6227d-739a-4f24-a633-ad3a3468bc19" (UID: "e1c6227d-739a-4f24-a633-ad3a3468bc19"). InnerVolumeSpecName "kube-api-access-nsfk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:09:49.232971 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.232959    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1c6227d-739a-4f24-a633-ad3a3468bc19-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e1c6227d-739a-4f24-a633-ad3a3468bc19" (UID: "e1c6227d-739a-4f24-a633-ad3a3468bc19"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:09:49.332774 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.332739    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1c6227d-739a-4f24-a633-ad3a3468bc19-openshift-service-ca\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:09:49.332774 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.332769    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsfk2\" (UniqueName: \"kubernetes.io/projected/e1c6227d-739a-4f24-a633-ad3a3468bc19-kube-api-access-nsfk2\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:09:49.332774 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.332779    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1c6227d-739a-4f24-a633-ad3a3468bc19-maas-api-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:09:49.724582 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.724547    2559 generic.go:356] "Generic (PLEG): container finished" podID="e1c6227d-739a-4f24-a633-ad3a3468bc19" containerID="34168c013c611409bce356ca737af1c0228b6fb9edcda9b936167082a0030855" exitCode=0
Jul 26 16:09:49.724582 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.724587    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"
Jul 26 16:09:49.724840 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.724625    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj" event={"ID":"e1c6227d-739a-4f24-a633-ad3a3468bc19","Type":"ContainerDied","Data":"34168c013c611409bce356ca737af1c0228b6fb9edcda9b936167082a0030855"}
Jul 26 16:09:49.724840 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.724676    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj" event={"ID":"e1c6227d-739a-4f24-a633-ad3a3468bc19","Type":"ContainerDied","Data":"89e548e149a6a34d7c73c68c71bd007128b91593c805f3f14a733b0de1e43cb1"}
Jul 26 16:09:49.724840 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.724699    2559 scope.go:117] "RemoveContainer" containerID="34168c013c611409bce356ca737af1c0228b6fb9edcda9b936167082a0030855"
Jul 26 16:09:49.734532 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.734511    2559 scope.go:117] "RemoveContainer" containerID="34168c013c611409bce356ca737af1c0228b6fb9edcda9b936167082a0030855"
Jul 26 16:09:49.734798 ip-10-0-139-110 kubenswrapper[2559]: E0726 16:09:49.734778    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34168c013c611409bce356ca737af1c0228b6fb9edcda9b936167082a0030855\": container with ID starting with 34168c013c611409bce356ca737af1c0228b6fb9edcda9b936167082a0030855 not found: ID does not exist" containerID="34168c013c611409bce356ca737af1c0228b6fb9edcda9b936167082a0030855"
Jul 26 16:09:49.734860 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.734810    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34168c013c611409bce356ca737af1c0228b6fb9edcda9b936167082a0030855"} err="failed to get container status \"34168c013c611409bce356ca737af1c0228b6fb9edcda9b936167082a0030855\": rpc error: code = NotFound desc = could not find container \"34168c013c611409bce356ca737af1c0228b6fb9edcda9b936167082a0030855\": container with ID starting with 34168c013c611409bce356ca737af1c0228b6fb9edcda9b936167082a0030855 not found: ID does not exist"
Jul 26 16:09:49.747225 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.747176    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"]
Jul 26 16:09:49.749719 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:49.749695    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-36235c-89cb84fc9-4cssj"]
Jul 26 16:09:50.194640 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:50.194607    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1c6227d-739a-4f24-a633-ad3a3468bc19" path="/var/lib/kubelet/pods/e1c6227d-739a-4f24-a633-ad3a3468bc19/volumes"
Jul 26 16:09:52.979386 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:52.979356    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:53.353260 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:53.353226    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:59.257636 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:59.257599    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:59.357942 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:59.357913    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:59.528247 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:59.528134    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:09:59.751781 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:09:59.751752    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:10:21.172407 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.172365    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"]
Jul 26 16:10:21.172924 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.172616    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87" podUID="1eaac55a-a153-4554-81ff-b721a20d006e" containerName="maas-api" containerID="cri-o://ad5aacc85d437f5e14f847fc4aa34dee026251aec0ea7d1197b73d7fde861734" gracePeriod=30
Jul 26 16:10:21.191109 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.191080    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:10:21.224688 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.224646    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8"]
Jul 26 16:10:21.224892 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.224865    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8" podUID="d8f87181-6f0e-4cae-9040-d3c7065971af" containerName="payload-processing" containerID="cri-o://6a9cc51233886d527e264bcaac112b6079d3bca4a3c9dcd449d091588aeecd80" gracePeriod=30
Jul 26 16:10:21.233005 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.232978    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs"]
Jul 26 16:10:21.233342 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.233308    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs" podUID="1f0209b3-90b9-46d4-b4c6-2d4a7eb87902" containerName="payload-pre-processing" containerID="cri-o://7a3d12ffeb2fa3adfb12e5b65ea1f5e5911dc9d2a47686767c8c407249d20c90" gracePeriod=30
Jul 26 16:10:21.265883 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.265858    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:10:21.328566 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.328530    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:10:21.535570 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.535544    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8"
Jul 26 16:10:21.538781 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.538762    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"
Jul 26 16:10:21.543995 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.543975    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs"
Jul 26 16:10:21.666510 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.666475    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zs54k\" (UniqueName: \"kubernetes.io/projected/1eaac55a-a153-4554-81ff-b721a20d006e-kube-api-access-zs54k\") pod \"1eaac55a-a153-4554-81ff-b721a20d006e\" (UID: \"1eaac55a-a153-4554-81ff-b721a20d006e\") "
Jul 26 16:10:21.666677 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.666525    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8f87181-6f0e-4cae-9040-d3c7065971af-plugins-config-volume\") pod \"d8f87181-6f0e-4cae-9040-d3c7065971af\" (UID: \"d8f87181-6f0e-4cae-9040-d3c7065971af\") "
Jul 26 16:10:21.666677 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.666562    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1eaac55a-a153-4554-81ff-b721a20d006e-openshift-service-ca\") pod \"1eaac55a-a153-4554-81ff-b721a20d006e\" (UID: \"1eaac55a-a153-4554-81ff-b721a20d006e\") "
Jul 26 16:10:21.666677 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.666614    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f0209b3-90b9-46d4-b4c6-2d4a7eb87902-plugins-config-volume\") pod \"1f0209b3-90b9-46d4-b4c6-2d4a7eb87902\" (UID: \"1f0209b3-90b9-46d4-b4c6-2d4a7eb87902\") "
Jul 26 16:10:21.666677 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.666664    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9b6jx\" (UniqueName: \"kubernetes.io/projected/d8f87181-6f0e-4cae-9040-d3c7065971af-kube-api-access-9b6jx\") pod \"d8f87181-6f0e-4cae-9040-d3c7065971af\" (UID: \"d8f87181-6f0e-4cae-9040-d3c7065971af\") "
Jul 26 16:10:21.666843 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.666740    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1eaac55a-a153-4554-81ff-b721a20d006e-maas-api-tls\") pod \"1eaac55a-a153-4554-81ff-b721a20d006e\" (UID: \"1eaac55a-a153-4554-81ff-b721a20d006e\") "
Jul 26 16:10:21.666843 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.666778    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjqm5\" (UniqueName: \"kubernetes.io/projected/1f0209b3-90b9-46d4-b4c6-2d4a7eb87902-kube-api-access-mjqm5\") pod \"1f0209b3-90b9-46d4-b4c6-2d4a7eb87902\" (UID: \"1f0209b3-90b9-46d4-b4c6-2d4a7eb87902\") "
Jul 26 16:10:21.667016 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.666989    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1eaac55a-a153-4554-81ff-b721a20d006e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1eaac55a-a153-4554-81ff-b721a20d006e" (UID: "1eaac55a-a153-4554-81ff-b721a20d006e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:10:21.667016 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.667002    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f0209b3-90b9-46d4-b4c6-2d4a7eb87902-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1f0209b3-90b9-46d4-b4c6-2d4a7eb87902" (UID: "1f0209b3-90b9-46d4-b4c6-2d4a7eb87902"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:10:21.667362 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.667338    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f0209b3-90b9-46d4-b4c6-2d4a7eb87902-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:10:21.667362 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.667351    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8f87181-6f0e-4cae-9040-d3c7065971af-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d8f87181-6f0e-4cae-9040-d3c7065971af" (UID: "d8f87181-6f0e-4cae-9040-d3c7065971af"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:10:21.667527 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.667365    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1eaac55a-a153-4554-81ff-b721a20d006e-openshift-service-ca\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:10:21.668871 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.668842    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1eaac55a-a153-4554-81ff-b721a20d006e-kube-api-access-zs54k" (OuterVolumeSpecName: "kube-api-access-zs54k") pod "1eaac55a-a153-4554-81ff-b721a20d006e" (UID: "1eaac55a-a153-4554-81ff-b721a20d006e"). InnerVolumeSpecName "kube-api-access-zs54k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:10:21.668871 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.668834    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f0209b3-90b9-46d4-b4c6-2d4a7eb87902-kube-api-access-mjqm5" (OuterVolumeSpecName: "kube-api-access-mjqm5") pod "1f0209b3-90b9-46d4-b4c6-2d4a7eb87902" (UID: "1f0209b3-90b9-46d4-b4c6-2d4a7eb87902"). InnerVolumeSpecName "kube-api-access-mjqm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:10:21.669000 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.668869    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1eaac55a-a153-4554-81ff-b721a20d006e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1eaac55a-a153-4554-81ff-b721a20d006e" (UID: "1eaac55a-a153-4554-81ff-b721a20d006e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:10:21.669000 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.668900    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8f87181-6f0e-4cae-9040-d3c7065971af-kube-api-access-9b6jx" (OuterVolumeSpecName: "kube-api-access-9b6jx") pod "d8f87181-6f0e-4cae-9040-d3c7065971af" (UID: "d8f87181-6f0e-4cae-9040-d3c7065971af"). InnerVolumeSpecName "kube-api-access-9b6jx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:10:21.768328 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.768247    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjqm5\" (UniqueName: \"kubernetes.io/projected/1f0209b3-90b9-46d4-b4c6-2d4a7eb87902-kube-api-access-mjqm5\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:10:21.768328 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.768275    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs54k\" (UniqueName: \"kubernetes.io/projected/1eaac55a-a153-4554-81ff-b721a20d006e-kube-api-access-zs54k\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:10:21.768328 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.768285    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8f87181-6f0e-4cae-9040-d3c7065971af-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:10:21.768328 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.768293    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9b6jx\" (UniqueName: \"kubernetes.io/projected/d8f87181-6f0e-4cae-9040-d3c7065971af-kube-api-access-9b6jx\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:10:21.768328 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.768304    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1eaac55a-a153-4554-81ff-b721a20d006e-maas-api-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:10:21.904361 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.904321    2559 generic.go:356] "Generic (PLEG): container finished" podID="1f0209b3-90b9-46d4-b4c6-2d4a7eb87902" containerID="7a3d12ffeb2fa3adfb12e5b65ea1f5e5911dc9d2a47686767c8c407249d20c90" exitCode=0
Jul 26 16:10:21.904533 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.904371    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs"
Jul 26 16:10:21.904533 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.904403    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs" event={"ID":"1f0209b3-90b9-46d4-b4c6-2d4a7eb87902","Type":"ContainerDied","Data":"7a3d12ffeb2fa3adfb12e5b65ea1f5e5911dc9d2a47686767c8c407249d20c90"}
Jul 26 16:10:21.904533 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.904461    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs" event={"ID":"1f0209b3-90b9-46d4-b4c6-2d4a7eb87902","Type":"ContainerDied","Data":"4110047e5d7f863b9351b03c0c8d6d2297a6c1d65f2c17feb27017d9b4adf0ba"}
Jul 26 16:10:21.904533 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.904493    2559 scope.go:117] "RemoveContainer" containerID="7a3d12ffeb2fa3adfb12e5b65ea1f5e5911dc9d2a47686767c8c407249d20c90"
Jul 26 16:10:21.905685 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.905664    2559 generic.go:356] "Generic (PLEG): container finished" podID="1eaac55a-a153-4554-81ff-b721a20d006e" containerID="ad5aacc85d437f5e14f847fc4aa34dee026251aec0ea7d1197b73d7fde861734" exitCode=0
Jul 26 16:10:21.905803 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.905700    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"
Jul 26 16:10:21.905803 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.905764    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87" event={"ID":"1eaac55a-a153-4554-81ff-b721a20d006e","Type":"ContainerDied","Data":"ad5aacc85d437f5e14f847fc4aa34dee026251aec0ea7d1197b73d7fde861734"}
Jul 26 16:10:21.905957 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.905821    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87" event={"ID":"1eaac55a-a153-4554-81ff-b721a20d006e","Type":"ContainerDied","Data":"066c01f05fb3626d7071c8bec548f0b6c35fc09641ae75734faa728478fdd3c5"}
Jul 26 16:10:21.907403 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.907382    2559 generic.go:356] "Generic (PLEG): container finished" podID="d8f87181-6f0e-4cae-9040-d3c7065971af" containerID="6a9cc51233886d527e264bcaac112b6079d3bca4a3c9dcd449d091588aeecd80" exitCode=0
Jul 26 16:10:21.907516 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.907426    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8"
Jul 26 16:10:21.907566 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.907429    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8" event={"ID":"d8f87181-6f0e-4cae-9040-d3c7065971af","Type":"ContainerDied","Data":"6a9cc51233886d527e264bcaac112b6079d3bca4a3c9dcd449d091588aeecd80"}
Jul 26 16:10:21.907608 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.907570    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8" event={"ID":"d8f87181-6f0e-4cae-9040-d3c7065971af","Type":"ContainerDied","Data":"2581d8c6037f402d5e6c84f7bf425090d611c360a1e6df8d62564763f1d64cfd"}
Jul 26 16:10:21.914415 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.914392    2559 scope.go:117] "RemoveContainer" containerID="7a3d12ffeb2fa3adfb12e5b65ea1f5e5911dc9d2a47686767c8c407249d20c90"
Jul 26 16:10:21.914769 ip-10-0-139-110 kubenswrapper[2559]: E0726 16:10:21.914742    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a3d12ffeb2fa3adfb12e5b65ea1f5e5911dc9d2a47686767c8c407249d20c90\": container with ID starting with 7a3d12ffeb2fa3adfb12e5b65ea1f5e5911dc9d2a47686767c8c407249d20c90 not found: ID does not exist" containerID="7a3d12ffeb2fa3adfb12e5b65ea1f5e5911dc9d2a47686767c8c407249d20c90"
Jul 26 16:10:21.914890 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.914780    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a3d12ffeb2fa3adfb12e5b65ea1f5e5911dc9d2a47686767c8c407249d20c90"} err="failed to get container status \"7a3d12ffeb2fa3adfb12e5b65ea1f5e5911dc9d2a47686767c8c407249d20c90\": rpc error: code = NotFound desc = could not find container \"7a3d12ffeb2fa3adfb12e5b65ea1f5e5911dc9d2a47686767c8c407249d20c90\": container with ID starting with 7a3d12ffeb2fa3adfb12e5b65ea1f5e5911dc9d2a47686767c8c407249d20c90 not found: ID does not exist"
Jul 26 16:10:21.914890 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.914804    2559 scope.go:117] "RemoveContainer" containerID="ad5aacc85d437f5e14f847fc4aa34dee026251aec0ea7d1197b73d7fde861734"
Jul 26 16:10:21.924911 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.924627    2559 scope.go:117] "RemoveContainer" containerID="ad5aacc85d437f5e14f847fc4aa34dee026251aec0ea7d1197b73d7fde861734"
Jul 26 16:10:21.924974 ip-10-0-139-110 kubenswrapper[2559]: E0726 16:10:21.924942    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad5aacc85d437f5e14f847fc4aa34dee026251aec0ea7d1197b73d7fde861734\": container with ID starting with ad5aacc85d437f5e14f847fc4aa34dee026251aec0ea7d1197b73d7fde861734 not found: ID does not exist" containerID="ad5aacc85d437f5e14f847fc4aa34dee026251aec0ea7d1197b73d7fde861734"
Jul 26 16:10:21.925021 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.924968    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad5aacc85d437f5e14f847fc4aa34dee026251aec0ea7d1197b73d7fde861734"} err="failed to get container status \"ad5aacc85d437f5e14f847fc4aa34dee026251aec0ea7d1197b73d7fde861734\": rpc error: code = NotFound desc = could not find container \"ad5aacc85d437f5e14f847fc4aa34dee026251aec0ea7d1197b73d7fde861734\": container with ID starting with ad5aacc85d437f5e14f847fc4aa34dee026251aec0ea7d1197b73d7fde861734 not found: ID does not exist"
Jul 26 16:10:21.925021 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.924990    2559 scope.go:117] "RemoveContainer" containerID="6a9cc51233886d527e264bcaac112b6079d3bca4a3c9dcd449d091588aeecd80"
Jul 26 16:10:21.927586 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.927469    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs"]
Jul 26 16:10:21.929706 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.929686    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-9a599d-566c485c68-h2mgs"]
Jul 26 16:10:21.933279 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.933262    2559 scope.go:117] "RemoveContainer" containerID="6a9cc51233886d527e264bcaac112b6079d3bca4a3c9dcd449d091588aeecd80"
Jul 26 16:10:21.933537 ip-10-0-139-110 kubenswrapper[2559]: E0726 16:10:21.933519    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a9cc51233886d527e264bcaac112b6079d3bca4a3c9dcd449d091588aeecd80\": container with ID starting with 6a9cc51233886d527e264bcaac112b6079d3bca4a3c9dcd449d091588aeecd80 not found: ID does not exist" containerID="6a9cc51233886d527e264bcaac112b6079d3bca4a3c9dcd449d091588aeecd80"
Jul 26 16:10:21.933600 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.933544    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a9cc51233886d527e264bcaac112b6079d3bca4a3c9dcd449d091588aeecd80"} err="failed to get container status \"6a9cc51233886d527e264bcaac112b6079d3bca4a3c9dcd449d091588aeecd80\": rpc error: code = NotFound desc = could not find container \"6a9cc51233886d527e264bcaac112b6079d3bca4a3c9dcd449d091588aeecd80\": container with ID starting with 6a9cc51233886d527e264bcaac112b6079d3bca4a3c9dcd449d091588aeecd80 not found: ID does not exist"
Jul 26 16:10:21.940156 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.940122    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"]
Jul 26 16:10:21.943827 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.943809    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-9a599d-5d6f954fc-z8d87"]
Jul 26 16:10:21.953558 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.953538    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8"]
Jul 26 16:10:21.959685 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:21.959665    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9a599d-c5b7cd5cb-59ft8"]
Jul 26 16:10:22.195058 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:22.195026    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1eaac55a-a153-4554-81ff-b721a20d006e" path="/var/lib/kubelet/pods/1eaac55a-a153-4554-81ff-b721a20d006e/volumes"
Jul 26 16:10:22.195483 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:22.195462    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f0209b3-90b9-46d4-b4c6-2d4a7eb87902" path="/var/lib/kubelet/pods/1f0209b3-90b9-46d4-b4c6-2d4a7eb87902/volumes"
Jul 26 16:10:22.195820 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:22.195807    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8f87181-6f0e-4cae-9040-d3c7065971af" path="/var/lib/kubelet/pods/d8f87181-6f0e-4cae-9040-d3c7065971af/volumes"
Jul 26 16:10:23.289226 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:23.289170    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:10:23.437593 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:23.437551    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:10:23.675418 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:23.675374    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:10:28.827693 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:28.827659    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:10:45.720182 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:45.720131    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:10:45.818790 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:10:45.818717    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:05.066754 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:05.066667    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:06.469712 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:06.469675    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:12.108370 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:12.108328    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:12.206556 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:12.206522    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:12.379105 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:12.379020    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:12.521446 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:12.521409    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:12.639676 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:12.639576    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:24.294265 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.294223    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"]
Jul 26 16:11:24.294678 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.294499    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc" podUID="f62eab5b-d00e-4abf-8054-b4a0272fd8c7" containerName="maas-api" containerID="cri-o://65b82c423c9951f618e80f84250eee834af26df2d6b866445ca73c820ab6d74f" gracePeriod=30
Jul 26 16:11:24.322635 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.322597    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:24.357110 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.357073    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl"]
Jul 26 16:11:24.357396 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.357361    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl" podUID="48c36a65-596e-4d6f-bd85-c60b581d9fcd" containerName="payload-processing" containerID="cri-o://3fff67e6bca45dc2794dbb42cef87ec8716fb3f6a3976b718f69119ef4c6e6bb" gracePeriod=30
Jul 26 16:11:24.364352 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.364320    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66"]
Jul 26 16:11:24.364610 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.364557    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66" podUID="10a45dbb-6a48-4633-8c59-ee9f6272151b" containerName="payload-pre-processing" containerID="cri-o://ed452b7803fc4bd38939dfb1d48d34c95d6348e732be5f6f85f0bd0287fac23e" gracePeriod=30
Jul 26 16:11:24.396426 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.396394    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:24.450109 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.450074    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:24.617179 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.617147    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"
Jul 26 16:11:24.656145 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.656112    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl"
Jul 26 16:11:24.688285 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.688256    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66"
Jul 26 16:11:24.703368 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.703338    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-maas-api-tls\") pod \"f62eab5b-d00e-4abf-8054-b4a0272fd8c7\" (UID: \"f62eab5b-d00e-4abf-8054-b4a0272fd8c7\") "
Jul 26 16:11:24.703978 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.703953    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh8r8\" (UniqueName: \"kubernetes.io/projected/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-kube-api-access-xh8r8\") pod \"f62eab5b-d00e-4abf-8054-b4a0272fd8c7\" (UID: \"f62eab5b-d00e-4abf-8054-b4a0272fd8c7\") "
Jul 26 16:11:24.704119 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.704001    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-openshift-service-ca\") pod \"f62eab5b-d00e-4abf-8054-b4a0272fd8c7\" (UID: \"f62eab5b-d00e-4abf-8054-b4a0272fd8c7\") "
Jul 26 16:11:24.704693 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.704667    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f62eab5b-d00e-4abf-8054-b4a0272fd8c7" (UID: "f62eab5b-d00e-4abf-8054-b4a0272fd8c7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:11:24.706103 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.706080    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f62eab5b-d00e-4abf-8054-b4a0272fd8c7" (UID: "f62eab5b-d00e-4abf-8054-b4a0272fd8c7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:11:24.706879 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.706858    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-kube-api-access-xh8r8" (OuterVolumeSpecName: "kube-api-access-xh8r8") pod "f62eab5b-d00e-4abf-8054-b4a0272fd8c7" (UID: "f62eab5b-d00e-4abf-8054-b4a0272fd8c7"). InnerVolumeSpecName "kube-api-access-xh8r8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:11:24.805640 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.805603    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlsbn\" (UniqueName: \"kubernetes.io/projected/48c36a65-596e-4d6f-bd85-c60b581d9fcd-kube-api-access-mlsbn\") pod \"48c36a65-596e-4d6f-bd85-c60b581d9fcd\" (UID: \"48c36a65-596e-4d6f-bd85-c60b581d9fcd\") "
Jul 26 16:11:24.805812 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.805700    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48c36a65-596e-4d6f-bd85-c60b581d9fcd-plugins-config-volume\") pod \"48c36a65-596e-4d6f-bd85-c60b581d9fcd\" (UID: \"48c36a65-596e-4d6f-bd85-c60b581d9fcd\") "
Jul 26 16:11:24.805812 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.805746    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jj9p\" (UniqueName: \"kubernetes.io/projected/10a45dbb-6a48-4633-8c59-ee9f6272151b-kube-api-access-2jj9p\") pod \"10a45dbb-6a48-4633-8c59-ee9f6272151b\" (UID: \"10a45dbb-6a48-4633-8c59-ee9f6272151b\") "
Jul 26 16:11:24.805812 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.805790    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10a45dbb-6a48-4633-8c59-ee9f6272151b-plugins-config-volume\") pod \"10a45dbb-6a48-4633-8c59-ee9f6272151b\" (UID: \"10a45dbb-6a48-4633-8c59-ee9f6272151b\") "
Jul 26 16:11:24.806118 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.806088    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48c36a65-596e-4d6f-bd85-c60b581d9fcd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "48c36a65-596e-4d6f-bd85-c60b581d9fcd" (UID: "48c36a65-596e-4d6f-bd85-c60b581d9fcd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:11:24.806242 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.806126    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10a45dbb-6a48-4633-8c59-ee9f6272151b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "10a45dbb-6a48-4633-8c59-ee9f6272151b" (UID: "10a45dbb-6a48-4633-8c59-ee9f6272151b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:11:24.806301 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.806284    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10a45dbb-6a48-4633-8c59-ee9f6272151b-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:11:24.806301 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.806298    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-maas-api-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:11:24.806367 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.806307    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh8r8\" (UniqueName: \"kubernetes.io/projected/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-kube-api-access-xh8r8\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:11:24.806367 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.806317    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f62eab5b-d00e-4abf-8054-b4a0272fd8c7-openshift-service-ca\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:11:24.806367 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.806326    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48c36a65-596e-4d6f-bd85-c60b581d9fcd-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:11:24.807957 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.807928    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10a45dbb-6a48-4633-8c59-ee9f6272151b-kube-api-access-2jj9p" (OuterVolumeSpecName: "kube-api-access-2jj9p") pod "10a45dbb-6a48-4633-8c59-ee9f6272151b" (UID: "10a45dbb-6a48-4633-8c59-ee9f6272151b"). InnerVolumeSpecName "kube-api-access-2jj9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:11:24.808065 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.807977    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48c36a65-596e-4d6f-bd85-c60b581d9fcd-kube-api-access-mlsbn" (OuterVolumeSpecName: "kube-api-access-mlsbn") pod "48c36a65-596e-4d6f-bd85-c60b581d9fcd" (UID: "48c36a65-596e-4d6f-bd85-c60b581d9fcd"). InnerVolumeSpecName "kube-api-access-mlsbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:11:24.907939 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.907846    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlsbn\" (UniqueName: \"kubernetes.io/projected/48c36a65-596e-4d6f-bd85-c60b581d9fcd-kube-api-access-mlsbn\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:11:24.907939 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:24.907874    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jj9p\" (UniqueName: \"kubernetes.io/projected/10a45dbb-6a48-4633-8c59-ee9f6272151b-kube-api-access-2jj9p\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:11:25.243987 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.243876    2559 generic.go:356] "Generic (PLEG): container finished" podID="48c36a65-596e-4d6f-bd85-c60b581d9fcd" containerID="3fff67e6bca45dc2794dbb42cef87ec8716fb3f6a3976b718f69119ef4c6e6bb" exitCode=0
Jul 26 16:11:25.243987 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.243925    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl"
Jul 26 16:11:25.243987 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.243966    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl" event={"ID":"48c36a65-596e-4d6f-bd85-c60b581d9fcd","Type":"ContainerDied","Data":"3fff67e6bca45dc2794dbb42cef87ec8716fb3f6a3976b718f69119ef4c6e6bb"}
Jul 26 16:11:25.244288 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.244022    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl" event={"ID":"48c36a65-596e-4d6f-bd85-c60b581d9fcd","Type":"ContainerDied","Data":"039a6d11ab68b286dee8aa918c84eec6621cb85420a592329e17ef5d7f19c766"}
Jul 26 16:11:25.244288 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.244051    2559 scope.go:117] "RemoveContainer" containerID="3fff67e6bca45dc2794dbb42cef87ec8716fb3f6a3976b718f69119ef4c6e6bb"
Jul 26 16:11:25.250484 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.250291    2559 generic.go:356] "Generic (PLEG): container finished" podID="f62eab5b-d00e-4abf-8054-b4a0272fd8c7" containerID="65b82c423c9951f618e80f84250eee834af26df2d6b866445ca73c820ab6d74f" exitCode=0
Jul 26 16:11:25.250484 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.250345    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"
Jul 26 16:11:25.250484 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.250411    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc" event={"ID":"f62eab5b-d00e-4abf-8054-b4a0272fd8c7","Type":"ContainerDied","Data":"65b82c423c9951f618e80f84250eee834af26df2d6b866445ca73c820ab6d74f"}
Jul 26 16:11:25.250923 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.250810    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc" event={"ID":"f62eab5b-d00e-4abf-8054-b4a0272fd8c7","Type":"ContainerDied","Data":"24ef0ef98d56c0b3b41a0c65364397f838b0a14850605e59c100e2dcf01b50d2"}
Jul 26 16:11:25.252597 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.252443    2559 generic.go:356] "Generic (PLEG): container finished" podID="10a45dbb-6a48-4633-8c59-ee9f6272151b" containerID="ed452b7803fc4bd38939dfb1d48d34c95d6348e732be5f6f85f0bd0287fac23e" exitCode=0
Jul 26 16:11:25.252597 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.252496    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66"
Jul 26 16:11:25.252597 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.252512    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66" event={"ID":"10a45dbb-6a48-4633-8c59-ee9f6272151b","Type":"ContainerDied","Data":"ed452b7803fc4bd38939dfb1d48d34c95d6348e732be5f6f85f0bd0287fac23e"}
Jul 26 16:11:25.252597 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.252545    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66" event={"ID":"10a45dbb-6a48-4633-8c59-ee9f6272151b","Type":"ContainerDied","Data":"7ada835e22a54f72bfb75100fad07ca190ec83d187ebea276edbb968a41561d4"}
Jul 26 16:11:25.262701 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.262561    2559 scope.go:117] "RemoveContainer" containerID="3fff67e6bca45dc2794dbb42cef87ec8716fb3f6a3976b718f69119ef4c6e6bb"
Jul 26 16:11:25.263120 ip-10-0-139-110 kubenswrapper[2559]: E0726 16:11:25.263094    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fff67e6bca45dc2794dbb42cef87ec8716fb3f6a3976b718f69119ef4c6e6bb\": container with ID starting with 3fff67e6bca45dc2794dbb42cef87ec8716fb3f6a3976b718f69119ef4c6e6bb not found: ID does not exist" containerID="3fff67e6bca45dc2794dbb42cef87ec8716fb3f6a3976b718f69119ef4c6e6bb"
Jul 26 16:11:25.263238 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.263132    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fff67e6bca45dc2794dbb42cef87ec8716fb3f6a3976b718f69119ef4c6e6bb"} err="failed to get container status \"3fff67e6bca45dc2794dbb42cef87ec8716fb3f6a3976b718f69119ef4c6e6bb\": rpc error: code = NotFound desc = could not find container \"3fff67e6bca45dc2794dbb42cef87ec8716fb3f6a3976b718f69119ef4c6e6bb\": container with ID starting with 3fff67e6bca45dc2794dbb42cef87ec8716fb3f6a3976b718f69119ef4c6e6bb not found: ID does not exist"
Jul 26 16:11:25.263238 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.263158    2559 scope.go:117] "RemoveContainer" containerID="65b82c423c9951f618e80f84250eee834af26df2d6b866445ca73c820ab6d74f"
Jul 26 16:11:25.268400 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.268377    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl"]
Jul 26 16:11:25.271912 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.271890    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-51789d-5fc65d8b68-4nlcl"]
Jul 26 16:11:25.273220 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.273182    2559 scope.go:117] "RemoveContainer" containerID="65b82c423c9951f618e80f84250eee834af26df2d6b866445ca73c820ab6d74f"
Jul 26 16:11:25.273970 ip-10-0-139-110 kubenswrapper[2559]: E0726 16:11:25.273575    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65b82c423c9951f618e80f84250eee834af26df2d6b866445ca73c820ab6d74f\": container with ID starting with 65b82c423c9951f618e80f84250eee834af26df2d6b866445ca73c820ab6d74f not found: ID does not exist" containerID="65b82c423c9951f618e80f84250eee834af26df2d6b866445ca73c820ab6d74f"
Jul 26 16:11:25.273970 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.273612    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65b82c423c9951f618e80f84250eee834af26df2d6b866445ca73c820ab6d74f"} err="failed to get container status \"65b82c423c9951f618e80f84250eee834af26df2d6b866445ca73c820ab6d74f\": rpc error: code = NotFound desc = could not find container \"65b82c423c9951f618e80f84250eee834af26df2d6b866445ca73c820ab6d74f\": container with ID starting with 65b82c423c9951f618e80f84250eee834af26df2d6b866445ca73c820ab6d74f not found: ID does not exist"
Jul 26 16:11:25.273970 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.273636    2559 scope.go:117] "RemoveContainer" containerID="ed452b7803fc4bd38939dfb1d48d34c95d6348e732be5f6f85f0bd0287fac23e"
Jul 26 16:11:25.283081 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.283054    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"]
Jul 26 16:11:25.285736 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.285708    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-51789d-755bd66cff-nlxmc"]
Jul 26 16:11:25.288783 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.288763    2559 scope.go:117] "RemoveContainer" containerID="ed452b7803fc4bd38939dfb1d48d34c95d6348e732be5f6f85f0bd0287fac23e"
Jul 26 16:11:25.289067 ip-10-0-139-110 kubenswrapper[2559]: E0726 16:11:25.289049    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed452b7803fc4bd38939dfb1d48d34c95d6348e732be5f6f85f0bd0287fac23e\": container with ID starting with ed452b7803fc4bd38939dfb1d48d34c95d6348e732be5f6f85f0bd0287fac23e not found: ID does not exist" containerID="ed452b7803fc4bd38939dfb1d48d34c95d6348e732be5f6f85f0bd0287fac23e"
Jul 26 16:11:25.289118 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.289076    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed452b7803fc4bd38939dfb1d48d34c95d6348e732be5f6f85f0bd0287fac23e"} err="failed to get container status \"ed452b7803fc4bd38939dfb1d48d34c95d6348e732be5f6f85f0bd0287fac23e\": rpc error: code = NotFound desc = could not find container \"ed452b7803fc4bd38939dfb1d48d34c95d6348e732be5f6f85f0bd0287fac23e\": container with ID starting with ed452b7803fc4bd38939dfb1d48d34c95d6348e732be5f6f85f0bd0287fac23e not found: ID does not exist"
Jul 26 16:11:25.294703 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.294681    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66"]
Jul 26 16:11:25.298072 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:25.298054    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-51789d-74dd67685b-t4s66"]
Jul 26 16:11:26.194780 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:26.194744    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10a45dbb-6a48-4633-8c59-ee9f6272151b" path="/var/lib/kubelet/pods/10a45dbb-6a48-4633-8c59-ee9f6272151b/volumes"
Jul 26 16:11:26.195122 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:26.195109    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48c36a65-596e-4d6f-bd85-c60b581d9fcd" path="/var/lib/kubelet/pods/48c36a65-596e-4d6f-bd85-c60b581d9fcd/volumes"
Jul 26 16:11:26.195467 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:26.195455    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f62eab5b-d00e-4abf-8054-b4a0272fd8c7" path="/var/lib/kubelet/pods/f62eab5b-d00e-4abf-8054-b4a0272fd8c7/volumes"
Jul 26 16:11:27.822550 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:27.822513    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:28.213798 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:28.213710    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:34.169272 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:34.169235    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:34.269949 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:34.269916    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:34.438866 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:34.438785    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:34.679733 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:34.679695    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:46.354638 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.354602    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"]
Jul 26 16:11:46.355149 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.354906    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b" podUID="ab134f70-a1a1-4f4d-bb02-21e08eb439b6" containerName="maas-api" containerID="cri-o://18c8433607d9cab0c9343c798567fd027af6de9e4c76b38178412c9c74f111a2" gracePeriod=30
Jul 26 16:11:46.388174 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.388133    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:46.420767 ip-10-0-139-110 kubenswrapper[2559]: E0726 16:11:46.420727    2559 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab134f70_a1a1_4f4d_bb02_21e08eb439b6.slice/crio-conmon-18c8433607d9cab0c9343c798567fd027af6de9e4c76b38178412c9c74f111a2.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab134f70_a1a1_4f4d_bb02_21e08eb439b6.slice/crio-18c8433607d9cab0c9343c798567fd027af6de9e4c76b38178412c9c74f111a2.scope\": RecentStats: unable to find data in memory cache]"
Jul 26 16:11:46.426225 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.426177    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb"]
Jul 26 16:11:46.426467 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.426419    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb" podUID="10017692-bc17-469f-ab16-ea63137a60b7" containerName="payload-pre-processing" containerID="cri-o://01bbcf347c4aa43d832d9a986bc50b23a501b59a1f2d562b3a4d05d955be2139" gracePeriod=30
Jul 26 16:11:46.481517 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.481482    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:46.579135 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.579100    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:46.670905 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.670879    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"
Jul 26 16:11:46.714530 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.714501    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb"
Jul 26 16:11:46.724903 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.724880    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab134f70-a1a1-4f4d-bb02-21e08eb439b6-openshift-service-ca\") pod \"ab134f70-a1a1-4f4d-bb02-21e08eb439b6\" (UID: \"ab134f70-a1a1-4f4d-bb02-21e08eb439b6\") "
Jul 26 16:11:46.725040 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.724994    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab134f70-a1a1-4f4d-bb02-21e08eb439b6-maas-api-tls\") pod \"ab134f70-a1a1-4f4d-bb02-21e08eb439b6\" (UID: \"ab134f70-a1a1-4f4d-bb02-21e08eb439b6\") "
Jul 26 16:11:46.725040 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.725036    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkwl9\" (UniqueName: \"kubernetes.io/projected/ab134f70-a1a1-4f4d-bb02-21e08eb439b6-kube-api-access-xkwl9\") pod \"ab134f70-a1a1-4f4d-bb02-21e08eb439b6\" (UID: \"ab134f70-a1a1-4f4d-bb02-21e08eb439b6\") "
Jul 26 16:11:46.725306 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.725279    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab134f70-a1a1-4f4d-bb02-21e08eb439b6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ab134f70-a1a1-4f4d-bb02-21e08eb439b6" (UID: "ab134f70-a1a1-4f4d-bb02-21e08eb439b6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:11:46.726965 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.726944    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab134f70-a1a1-4f4d-bb02-21e08eb439b6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ab134f70-a1a1-4f4d-bb02-21e08eb439b6" (UID: "ab134f70-a1a1-4f4d-bb02-21e08eb439b6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:11:46.727089 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.727070    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab134f70-a1a1-4f4d-bb02-21e08eb439b6-kube-api-access-xkwl9" (OuterVolumeSpecName: "kube-api-access-xkwl9") pod "ab134f70-a1a1-4f4d-bb02-21e08eb439b6" (UID: "ab134f70-a1a1-4f4d-bb02-21e08eb439b6"). InnerVolumeSpecName "kube-api-access-xkwl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:11:46.826053 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.826014    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wf2kp\" (UniqueName: \"kubernetes.io/projected/10017692-bc17-469f-ab16-ea63137a60b7-kube-api-access-wf2kp\") pod \"10017692-bc17-469f-ab16-ea63137a60b7\" (UID: \"10017692-bc17-469f-ab16-ea63137a60b7\") "
Jul 26 16:11:46.826264 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.826091    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10017692-bc17-469f-ab16-ea63137a60b7-plugins-config-volume\") pod \"10017692-bc17-469f-ab16-ea63137a60b7\" (UID: \"10017692-bc17-469f-ab16-ea63137a60b7\") "
Jul 26 16:11:46.826521 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.826493    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab134f70-a1a1-4f4d-bb02-21e08eb439b6-maas-api-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:11:46.826521 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.826521    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkwl9\" (UniqueName: \"kubernetes.io/projected/ab134f70-a1a1-4f4d-bb02-21e08eb439b6-kube-api-access-xkwl9\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:11:46.826709 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.826525    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10017692-bc17-469f-ab16-ea63137a60b7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "10017692-bc17-469f-ab16-ea63137a60b7" (UID: "10017692-bc17-469f-ab16-ea63137a60b7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:11:46.826709 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.826535    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab134f70-a1a1-4f4d-bb02-21e08eb439b6-openshift-service-ca\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:11:46.828102 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.828080    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10017692-bc17-469f-ab16-ea63137a60b7-kube-api-access-wf2kp" (OuterVolumeSpecName: "kube-api-access-wf2kp") pod "10017692-bc17-469f-ab16-ea63137a60b7" (UID: "10017692-bc17-469f-ab16-ea63137a60b7"). InnerVolumeSpecName "kube-api-access-wf2kp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:11:46.927816 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.927706    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wf2kp\" (UniqueName: \"kubernetes.io/projected/10017692-bc17-469f-ab16-ea63137a60b7-kube-api-access-wf2kp\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:11:46.927816 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:46.927756    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10017692-bc17-469f-ab16-ea63137a60b7-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:11:47.379107 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:47.379076    2559 generic.go:356] "Generic (PLEG): container finished" podID="ab134f70-a1a1-4f4d-bb02-21e08eb439b6" containerID="18c8433607d9cab0c9343c798567fd027af6de9e4c76b38178412c9c74f111a2" exitCode=0
Jul 26 16:11:47.379529 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:47.379124    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"
Jul 26 16:11:47.379529 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:47.379157    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b" event={"ID":"ab134f70-a1a1-4f4d-bb02-21e08eb439b6","Type":"ContainerDied","Data":"18c8433607d9cab0c9343c798567fd027af6de9e4c76b38178412c9c74f111a2"}
Jul 26 16:11:47.379529 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:47.379232    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b" event={"ID":"ab134f70-a1a1-4f4d-bb02-21e08eb439b6","Type":"ContainerDied","Data":"10342cba64c6b73a1af1b18d10535381e9b14b3415a7446273d841a15cbf82c4"}
Jul 26 16:11:47.379529 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:47.379257    2559 scope.go:117] "RemoveContainer" containerID="18c8433607d9cab0c9343c798567fd027af6de9e4c76b38178412c9c74f111a2"
Jul 26 16:11:47.380951 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:47.380930    2559 generic.go:356] "Generic (PLEG): container finished" podID="10017692-bc17-469f-ab16-ea63137a60b7" containerID="01bbcf347c4aa43d832d9a986bc50b23a501b59a1f2d562b3a4d05d955be2139" exitCode=0
Jul 26 16:11:47.381035 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:47.380974    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb"
Jul 26 16:11:47.381035 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:47.381012    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb" event={"ID":"10017692-bc17-469f-ab16-ea63137a60b7","Type":"ContainerDied","Data":"01bbcf347c4aa43d832d9a986bc50b23a501b59a1f2d562b3a4d05d955be2139"}
Jul 26 16:11:47.381117 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:47.381063    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb" event={"ID":"10017692-bc17-469f-ab16-ea63137a60b7","Type":"ContainerDied","Data":"ccfb822e516eb2afddd6c6abbe6ba03e3199903a6de1fd26f29273027926b7ba"}
Jul 26 16:11:47.392404 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:47.389027    2559 scope.go:117] "RemoveContainer" containerID="18c8433607d9cab0c9343c798567fd027af6de9e4c76b38178412c9c74f111a2"
Jul 26 16:11:47.392752 ip-10-0-139-110 kubenswrapper[2559]: E0726 16:11:47.392558    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18c8433607d9cab0c9343c798567fd027af6de9e4c76b38178412c9c74f111a2\": container with ID starting with 18c8433607d9cab0c9343c798567fd027af6de9e4c76b38178412c9c74f111a2 not found: ID does not exist" containerID="18c8433607d9cab0c9343c798567fd027af6de9e4c76b38178412c9c74f111a2"
Jul 26 16:11:47.392752 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:47.392595    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18c8433607d9cab0c9343c798567fd027af6de9e4c76b38178412c9c74f111a2"} err="failed to get container status \"18c8433607d9cab0c9343c798567fd027af6de9e4c76b38178412c9c74f111a2\": rpc error: code = NotFound desc = could not find container \"18c8433607d9cab0c9343c798567fd027af6de9e4c76b38178412c9c74f111a2\": container with ID starting with 18c8433607d9cab0c9343c798567fd027af6de9e4c76b38178412c9c74f111a2 not found: ID does not exist"
Jul 26 16:11:47.392752 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:47.392621    2559 scope.go:117] "RemoveContainer" containerID="01bbcf347c4aa43d832d9a986bc50b23a501b59a1f2d562b3a4d05d955be2139"
Jul 26 16:11:47.405308 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:47.405266    2559 scope.go:117] "RemoveContainer" containerID="01bbcf347c4aa43d832d9a986bc50b23a501b59a1f2d562b3a4d05d955be2139"
Jul 26 16:11:47.405826 ip-10-0-139-110 kubenswrapper[2559]: E0726 16:11:47.405739    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01bbcf347c4aa43d832d9a986bc50b23a501b59a1f2d562b3a4d05d955be2139\": container with ID starting with 01bbcf347c4aa43d832d9a986bc50b23a501b59a1f2d562b3a4d05d955be2139 not found: ID does not exist" containerID="01bbcf347c4aa43d832d9a986bc50b23a501b59a1f2d562b3a4d05d955be2139"
Jul 26 16:11:47.405922 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:47.405848    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01bbcf347c4aa43d832d9a986bc50b23a501b59a1f2d562b3a4d05d955be2139"} err="failed to get container status \"01bbcf347c4aa43d832d9a986bc50b23a501b59a1f2d562b3a4d05d955be2139\": rpc error: code = NotFound desc = could not find container \"01bbcf347c4aa43d832d9a986bc50b23a501b59a1f2d562b3a4d05d955be2139\": container with ID starting with 01bbcf347c4aa43d832d9a986bc50b23a501b59a1f2d562b3a4d05d955be2139 not found: ID does not exist"
Jul 26 16:11:47.408462 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:47.408440    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"]
Jul 26 16:11:47.412222 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:47.412185    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d81393-5fcdbb865d-kvl7b"]
Jul 26 16:11:47.421459 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:47.421432    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb"]
Jul 26 16:11:47.426520 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:47.426496    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d81393-8454ffd548-9sphb"]
Jul 26 16:11:48.195002 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:48.194964    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10017692-bc17-469f-ab16-ea63137a60b7" path="/var/lib/kubelet/pods/10017692-bc17-469f-ab16-ea63137a60b7/volumes"
Jul 26 16:11:48.195414 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:48.195400    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab134f70-a1a1-4f4d-bb02-21e08eb439b6" path="/var/lib/kubelet/pods/ab134f70-a1a1-4f4d-bb02-21e08eb439b6/volumes"
Jul 26 16:11:49.886792 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:49.886753    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:50.252486 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:50.252393    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:56.416718 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:56.416685    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:56.517322 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:56.517291    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:56.706695 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:56.706614    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:56.922528 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:56.922494    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:11:59.680232 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:11:59.680180    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:12:07.276450 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.276413    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"]
Jul 26 16:12:07.276951 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.276676    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r" podUID="26450585-ef16-4808-b7a5-051056852cc7" containerName="maas-api" containerID="cri-o://ddc76576b63bccc36d02782ac70200a7e178855e82abcda75057c0d0f5e9845a" gracePeriod=30
Jul 26 16:12:07.304989 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.304918    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:12:07.340893 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.340860    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526"]
Jul 26 16:12:07.341108 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.341081    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526" podUID="eea021bd-a122-463e-b5b0-eb3de4ba9adc" containerName="payload-processing" containerID="cri-o://099a70ce044defbab34f2a57c7f18ffd690a7c405366bbe61eac7c525b400eaa" gracePeriod=30
Jul 26 16:12:07.391716 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.391681    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:12:07.447815 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.447784    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:12:07.489372 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.489342    2559 generic.go:356] "Generic (PLEG): container finished" podID="26450585-ef16-4808-b7a5-051056852cc7" containerID="ddc76576b63bccc36d02782ac70200a7e178855e82abcda75057c0d0f5e9845a" exitCode=0
Jul 26 16:12:07.489534 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.489467    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r" event={"ID":"26450585-ef16-4808-b7a5-051056852cc7","Type":"ContainerDied","Data":"ddc76576b63bccc36d02782ac70200a7e178855e82abcda75057c0d0f5e9845a"}
Jul 26 16:12:07.491289 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.491259    2559 generic.go:356] "Generic (PLEG): container finished" podID="eea021bd-a122-463e-b5b0-eb3de4ba9adc" containerID="099a70ce044defbab34f2a57c7f18ffd690a7c405366bbe61eac7c525b400eaa" exitCode=0
Jul 26 16:12:07.491430 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.491362    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526" event={"ID":"eea021bd-a122-463e-b5b0-eb3de4ba9adc","Type":"ContainerDied","Data":"099a70ce044defbab34f2a57c7f18ffd690a7c405366bbe61eac7c525b400eaa"}
Jul 26 16:12:07.597825 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.597793    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"
Jul 26 16:12:07.631322 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.631291    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526"
Jul 26 16:12:07.725426 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.725388    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eea021bd-a122-463e-b5b0-eb3de4ba9adc-plugins-config-volume\") pod \"eea021bd-a122-463e-b5b0-eb3de4ba9adc\" (UID: \"eea021bd-a122-463e-b5b0-eb3de4ba9adc\") "
Jul 26 16:12:07.725426 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.725430    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26450585-ef16-4808-b7a5-051056852cc7-openshift-service-ca\") pod \"26450585-ef16-4808-b7a5-051056852cc7\" (UID: \"26450585-ef16-4808-b7a5-051056852cc7\") "
Jul 26 16:12:07.725690 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.725526    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxbhs\" (UniqueName: \"kubernetes.io/projected/eea021bd-a122-463e-b5b0-eb3de4ba9adc-kube-api-access-sxbhs\") pod \"eea021bd-a122-463e-b5b0-eb3de4ba9adc\" (UID: \"eea021bd-a122-463e-b5b0-eb3de4ba9adc\") "
Jul 26 16:12:07.725690 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.725586    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26450585-ef16-4808-b7a5-051056852cc7-maas-api-tls\") pod \"26450585-ef16-4808-b7a5-051056852cc7\" (UID: \"26450585-ef16-4808-b7a5-051056852cc7\") "
Jul 26 16:12:07.725690 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.725647    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4f8q\" (UniqueName: \"kubernetes.io/projected/26450585-ef16-4808-b7a5-051056852cc7-kube-api-access-v4f8q\") pod \"26450585-ef16-4808-b7a5-051056852cc7\" (UID: \"26450585-ef16-4808-b7a5-051056852cc7\") "
Jul 26 16:12:07.725885 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.725845    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26450585-ef16-4808-b7a5-051056852cc7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "26450585-ef16-4808-b7a5-051056852cc7" (UID: "26450585-ef16-4808-b7a5-051056852cc7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:12:07.725955 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.725928    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eea021bd-a122-463e-b5b0-eb3de4ba9adc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eea021bd-a122-463e-b5b0-eb3de4ba9adc" (UID: "eea021bd-a122-463e-b5b0-eb3de4ba9adc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:12:07.726042 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.726015    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eea021bd-a122-463e-b5b0-eb3de4ba9adc-plugins-config-volume\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:12:07.726101 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.726041    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26450585-ef16-4808-b7a5-051056852cc7-openshift-service-ca\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:12:07.727794 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.727764    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26450585-ef16-4808-b7a5-051056852cc7-kube-api-access-v4f8q" (OuterVolumeSpecName: "kube-api-access-v4f8q") pod "26450585-ef16-4808-b7a5-051056852cc7" (UID: "26450585-ef16-4808-b7a5-051056852cc7"). InnerVolumeSpecName "kube-api-access-v4f8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:12:07.728239 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.728190    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26450585-ef16-4808-b7a5-051056852cc7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "26450585-ef16-4808-b7a5-051056852cc7" (UID: "26450585-ef16-4808-b7a5-051056852cc7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:12:07.728239 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.728235    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eea021bd-a122-463e-b5b0-eb3de4ba9adc-kube-api-access-sxbhs" (OuterVolumeSpecName: "kube-api-access-sxbhs") pod "eea021bd-a122-463e-b5b0-eb3de4ba9adc" (UID: "eea021bd-a122-463e-b5b0-eb3de4ba9adc"). InnerVolumeSpecName "kube-api-access-sxbhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:12:07.827178 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.827143    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26450585-ef16-4808-b7a5-051056852cc7-maas-api-tls\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:12:07.827178 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.827170    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4f8q\" (UniqueName: \"kubernetes.io/projected/26450585-ef16-4808-b7a5-051056852cc7-kube-api-access-v4f8q\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:12:07.827178 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:07.827180    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxbhs\" (UniqueName: \"kubernetes.io/projected/eea021bd-a122-463e-b5b0-eb3de4ba9adc-kube-api-access-sxbhs\") on node \"ip-10-0-139-110.ec2.internal\" DevicePath \"\""
Jul 26 16:12:08.497264 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:08.497145    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"
Jul 26 16:12:08.497264 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:08.497154    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r" event={"ID":"26450585-ef16-4808-b7a5-051056852cc7","Type":"ContainerDied","Data":"a219bdd6babaeb66a2199f36a492bb6b8d1d16f7a55753e4ccec607acc90990e"}
Jul 26 16:12:08.497757 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:08.497283    2559 scope.go:117] "RemoveContainer" containerID="ddc76576b63bccc36d02782ac70200a7e178855e82abcda75057c0d0f5e9845a"
Jul 26 16:12:08.498694 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:08.498595    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526" event={"ID":"eea021bd-a122-463e-b5b0-eb3de4ba9adc","Type":"ContainerDied","Data":"4029fa72e49b35f0d07f907e8f7673eaf2b1cd7021c7061b70374b96a52cda4c"}
Jul 26 16:12:08.498694 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:08.498632    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526"
Jul 26 16:12:08.506314 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:08.506290    2559 scope.go:117] "RemoveContainer" containerID="099a70ce044defbab34f2a57c7f18ffd690a7c405366bbe61eac7c525b400eaa"
Jul 26 16:12:08.515418 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:08.515397    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"]
Jul 26 16:12:08.520586 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:08.520560    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-976e7c-6b6d9846d6-rcg4r"]
Jul 26 16:12:08.529796 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:08.529768    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526"]
Jul 26 16:12:08.535247 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:08.535221    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-976e7c-645c6d8766-fw526"]
Jul 26 16:12:10.194087 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:10.194043    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26450585-ef16-4808-b7a5-051056852cc7" path="/var/lib/kubelet/pods/26450585-ef16-4808-b7a5-051056852cc7/volumes"
Jul 26 16:12:10.194620 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:10.194600    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eea021bd-a122-463e-b5b0-eb3de4ba9adc" path="/var/lib/kubelet/pods/eea021bd-a122-463e-b5b0-eb3de4ba9adc/volumes"
Jul 26 16:12:12.149005 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:12.148974    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:12:12.557774 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:12.557688    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:12:18.271127 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:18.271093    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:12:18.335403 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:18.335369    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:12:18.533535 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:18.533453    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:12:18.753436 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:18.753403    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:12:24.874634 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:24.874601    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-84d47fc9db-j89xf_9b703ae5-cd78-4b6f-874c-240eee156343/manager/0.log"
Jul 26 16:12:25.364080 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:25.364049    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79d786d858-cz6m4_b9601f1f-c7fc-41a8-9336-c44781094551/copy-manifests/0.log"
Jul 26 16:12:25.376452 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:25.376426    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79d786d858-cz6m4_b9601f1f-c7fc-41a8-9336-c44781094551/manager/0.log"
Jul 26 16:12:25.846245 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:25.846217    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6978d97978-hh2xg_c1c3b4cc-199c-4612-9b7d-9ff5273b7efe/maas-api/0.log"
Jul 26 16:12:26.190604 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:26.190512    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:12:26.308657 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:26.308629    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-bq82x_2bdd4663-f066-449f-ade8-ce0b25d36926/postgres/0.log"
Jul 26 16:12:27.920220 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:27.920124    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-zdpp9_4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef/kuadrant-console-plugin/0.log"
Jul 26 16:12:29.002099 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:29.002066    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dc54b99bb-9tw42_1afad906-4350-4409-a472-c8b30df96f7c/kube-auth-proxy/0.log"
Jul 26 16:12:29.358329 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:29.358302    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-vc5dp_58a004c8-8db2-46f4-851a-fa381aa8024a/payload-pre-processing/0.log"
Jul 26 16:12:29.477716 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:29.477689    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wsp9v_afc1d831-651b-4136-8a8e-8440f9a966c3/payload-processing/0.log"
Jul 26 16:12:29.945040 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:29.945007    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-x949x_1c73a48a-0e06-4de7-bd2a-9fb758ab4289/main/0.log"
Jul 26 16:12:30.453718 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:30.453685    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-qm6ms_7a47137a-5f31-4db0-ada4-1f9f3741c618/main/0.log"
Jul 26 16:12:35.885478 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.885440    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jncg9/must-gather-vzcxj"]
Jul 26 16:12:35.886027 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886007    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1c6227d-739a-4f24-a633-ad3a3468bc19" containerName="maas-api"
Jul 26 16:12:35.886104 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886029    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1c6227d-739a-4f24-a633-ad3a3468bc19" containerName="maas-api"
Jul 26 16:12:35.886104 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886045    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab134f70-a1a1-4f4d-bb02-21e08eb439b6" containerName="maas-api"
Jul 26 16:12:35.886104 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886053    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab134f70-a1a1-4f4d-bb02-21e08eb439b6" containerName="maas-api"
Jul 26 16:12:35.886104 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886076    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8f87181-6f0e-4cae-9040-d3c7065971af" containerName="payload-processing"
Jul 26 16:12:35.886104 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886084    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8f87181-6f0e-4cae-9040-d3c7065971af" containerName="payload-processing"
Jul 26 16:12:35.886104 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886094    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f62eab5b-d00e-4abf-8054-b4a0272fd8c7" containerName="maas-api"
Jul 26 16:12:35.886104 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886102    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f62eab5b-d00e-4abf-8054-b4a0272fd8c7" containerName="maas-api"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886117    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10a45dbb-6a48-4633-8c59-ee9f6272151b" containerName="payload-pre-processing"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886127    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="10a45dbb-6a48-4633-8c59-ee9f6272151b" containerName="payload-pre-processing"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886147    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eea021bd-a122-463e-b5b0-eb3de4ba9adc" containerName="payload-processing"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886155    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="eea021bd-a122-463e-b5b0-eb3de4ba9adc" containerName="payload-processing"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886164    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26450585-ef16-4808-b7a5-051056852cc7" containerName="maas-api"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886172    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="26450585-ef16-4808-b7a5-051056852cc7" containerName="maas-api"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886186    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10017692-bc17-469f-ab16-ea63137a60b7" containerName="payload-pre-processing"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886196    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="10017692-bc17-469f-ab16-ea63137a60b7" containerName="payload-pre-processing"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886221    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48c36a65-596e-4d6f-bd85-c60b581d9fcd" containerName="payload-processing"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886230    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="48c36a65-596e-4d6f-bd85-c60b581d9fcd" containerName="payload-processing"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886242    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f0209b3-90b9-46d4-b4c6-2d4a7eb87902" containerName="payload-pre-processing"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886250    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f0209b3-90b9-46d4-b4c6-2d4a7eb87902" containerName="payload-pre-processing"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886259    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1eaac55a-a153-4554-81ff-b721a20d006e" containerName="maas-api"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886268    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1eaac55a-a153-4554-81ff-b721a20d006e" containerName="maas-api"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886364    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f62eab5b-d00e-4abf-8054-b4a0272fd8c7" containerName="maas-api"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886376    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="10a45dbb-6a48-4633-8c59-ee9f6272151b" containerName="payload-pre-processing"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886388    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1c6227d-739a-4f24-a633-ad3a3468bc19" containerName="maas-api"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886399    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="26450585-ef16-4808-b7a5-051056852cc7" containerName="maas-api"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886407    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1eaac55a-a153-4554-81ff-b721a20d006e" containerName="maas-api"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886416    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f0209b3-90b9-46d4-b4c6-2d4a7eb87902" containerName="payload-pre-processing"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886425    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="48c36a65-596e-4d6f-bd85-c60b581d9fcd" containerName="payload-processing"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886435    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="10017692-bc17-469f-ab16-ea63137a60b7" containerName="payload-pre-processing"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886444    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="eea021bd-a122-463e-b5b0-eb3de4ba9adc" containerName="payload-processing"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886456    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8f87181-6f0e-4cae-9040-d3c7065971af" containerName="payload-processing"
Jul 26 16:12:35.886492 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.886466    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab134f70-a1a1-4f4d-bb02-21e08eb439b6" containerName="maas-api"
Jul 26 16:12:35.889965 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.889945    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jncg9/must-gather-vzcxj"
Jul 26 16:12:35.891931 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.891908    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jncg9\"/\"openshift-service-ca.crt\""
Jul 26 16:12:35.892037 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.891953    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jncg9\"/\"kube-root-ca.crt\""
Jul 26 16:12:35.892097 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.892030    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jncg9\"/\"default-dockercfg-km9tz\""
Jul 26 16:12:35.895709 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.895684    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jncg9/must-gather-vzcxj"]
Jul 26 16:12:35.984997 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.984968    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/81d25f41-59f1-4c7c-93d3-6967bcdda531-must-gather-output\") pod \"must-gather-vzcxj\" (UID: \"81d25f41-59f1-4c7c-93d3-6967bcdda531\") " pod="openshift-must-gather-jncg9/must-gather-vzcxj"
Jul 26 16:12:35.985171 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:35.985033    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfk2k\" (UniqueName: \"kubernetes.io/projected/81d25f41-59f1-4c7c-93d3-6967bcdda531-kube-api-access-rfk2k\") pod \"must-gather-vzcxj\" (UID: \"81d25f41-59f1-4c7c-93d3-6967bcdda531\") " pod="openshift-must-gather-jncg9/must-gather-vzcxj"
Jul 26 16:12:36.086644 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:36.086609    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfk2k\" (UniqueName: \"kubernetes.io/projected/81d25f41-59f1-4c7c-93d3-6967bcdda531-kube-api-access-rfk2k\") pod \"must-gather-vzcxj\" (UID: \"81d25f41-59f1-4c7c-93d3-6967bcdda531\") " pod="openshift-must-gather-jncg9/must-gather-vzcxj"
Jul 26 16:12:36.086839 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:36.086705    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/81d25f41-59f1-4c7c-93d3-6967bcdda531-must-gather-output\") pod \"must-gather-vzcxj\" (UID: \"81d25f41-59f1-4c7c-93d3-6967bcdda531\") " pod="openshift-must-gather-jncg9/must-gather-vzcxj"
Jul 26 16:12:36.087012 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:36.086997    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/81d25f41-59f1-4c7c-93d3-6967bcdda531-must-gather-output\") pod \"must-gather-vzcxj\" (UID: \"81d25f41-59f1-4c7c-93d3-6967bcdda531\") " pod="openshift-must-gather-jncg9/must-gather-vzcxj"
Jul 26 16:12:36.093784 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:36.093759    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jncg9\"/\"kube-root-ca.crt\""
Jul 26 16:12:36.104138 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:36.104109    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jncg9\"/\"openshift-service-ca.crt\""
Jul 26 16:12:36.114778 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:36.114752    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfk2k\" (UniqueName: \"kubernetes.io/projected/81d25f41-59f1-4c7c-93d3-6967bcdda531-kube-api-access-rfk2k\") pod \"must-gather-vzcxj\" (UID: \"81d25f41-59f1-4c7c-93d3-6967bcdda531\") " pod="openshift-must-gather-jncg9/must-gather-vzcxj"
Jul 26 16:12:36.202626 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:36.202553    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jncg9\"/\"default-dockercfg-km9tz\""
Jul 26 16:12:36.211179 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:36.211157    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jncg9/must-gather-vzcxj"
Jul 26 16:12:36.347244 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:36.347197    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jncg9/must-gather-vzcxj"]
Jul 26 16:12:36.349594 ip-10-0-139-110 kubenswrapper[2559]: W0726 16:12:36.349564    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81d25f41_59f1_4c7c_93d3_6967bcdda531.slice/crio-e3b45a1ecac7572a018ce8f2984ac48cfda311d56bbd12a08bc758e88bc2eb79 WatchSource:0}: Error finding container e3b45a1ecac7572a018ce8f2984ac48cfda311d56bbd12a08bc758e88bc2eb79: Status 404 returned error can't find the container with id e3b45a1ecac7572a018ce8f2984ac48cfda311d56bbd12a08bc758e88bc2eb79
Jul 26 16:12:36.351311 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:36.351294    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 16:12:36.484245 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:36.484143    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:12:36.585244 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:36.585211    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:12:36.641900 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:36.641866    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jncg9/must-gather-vzcxj" event={"ID":"81d25f41-59f1-4c7c-93d3-6967bcdda531","Type":"ContainerStarted","Data":"e3b45a1ecac7572a018ce8f2984ac48cfda311d56bbd12a08bc758e88bc2eb79"}
Jul 26 16:12:37.651236 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:37.651190    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jncg9/must-gather-vzcxj" event={"ID":"81d25f41-59f1-4c7c-93d3-6967bcdda531","Type":"ContainerStarted","Data":"54fbadf0beafbfc8c69cd331baee38e389e62feafa7de2598574672b6dcedcc4"}
Jul 26 16:12:37.651639 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:37.651249    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jncg9/must-gather-vzcxj" event={"ID":"81d25f41-59f1-4c7c-93d3-6967bcdda531","Type":"ContainerStarted","Data":"8b955ac63546434a2f0a70ab343e2c7a9e8d7db812154443816796d692925493"}
Jul 26 16:12:37.666446 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:37.666394    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jncg9/must-gather-vzcxj" podStartSLOduration=1.9057756380000002 podStartE2EDuration="2.666376878s" totalImagesPullingTime="760.60124ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:12:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:12:37.664722554 +0000 UTC m=+3241.954554675" watchObservedRunningTime="2026-07-26 16:12:37.666376878 +0000 UTC m=+3241.956208997"
Jul 26 16:12:38.602479 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:38.602426    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6g6lj_5645fd38-99a5-4fa7-bf50-bf044c0444b6/global-pull-secret-syncer/0.log"
Jul 26 16:12:38.736362 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:38.736335    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log"
Jul 26 16:12:38.887795 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:38.887728    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-110.ec2.internal_58d8a889e41c1cf708108977adf7b177/haproxy/0.log"
Jul 26 16:12:44.455399 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:44.455315    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-zdpp9_4c1d1bc3-44e5-4b22-9d10-726d0af0c9ef/kuadrant-console-plugin/0.log"
Jul 26 16:12:46.538689 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:46.538613    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_19d25697-6503-4a21-ac72-21388c1eda33/alertmanager/0.log"
Jul 26 16:12:46.559334 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:46.559305    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_19d25697-6503-4a21-ac72-21388c1eda33/config-reloader/0.log"
Jul 26 16:12:46.578930 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:46.578900    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_19d25697-6503-4a21-ac72-21388c1eda33/kube-rbac-proxy-web/0.log"
Jul 26 16:12:46.604110 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:46.604083    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_19d25697-6503-4a21-ac72-21388c1eda33/kube-rbac-proxy/0.log"
Jul 26 16:12:46.626165 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:46.626137    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_19d25697-6503-4a21-ac72-21388c1eda33/kube-rbac-proxy-metric/0.log"
Jul 26 16:12:46.650380 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:46.650356    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_19d25697-6503-4a21-ac72-21388c1eda33/prom-label-proxy/0.log"
Jul 26 16:12:46.673050 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:46.673019    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_19d25697-6503-4a21-ac72-21388c1eda33/init-config-reloader/0.log"
Jul 26 16:12:46.757356 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:46.757303    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-7h29h_70542744-fd1a-4471-b33d-80747ea33774/kube-state-metrics/0.log"
Jul 26 16:12:46.772299 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:46.772257    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-7h29h_70542744-fd1a-4471-b33d-80747ea33774/kube-rbac-proxy-main/0.log"
Jul 26 16:12:46.791489 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:46.791411    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-7h29h_70542744-fd1a-4471-b33d-80747ea33774/kube-rbac-proxy-self/0.log"
Jul 26 16:12:46.819545 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:46.819471    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-775b87ff9b-hzhj9_e2e42628-3aa6-421f-b3c5-06ed360daa81/metrics-server/0.log"
Jul 26 16:12:46.848104 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:46.848073    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-vtjpr_57e0df8f-7130-481e-998b-89f6e93e8b01/monitoring-plugin/0.log"
Jul 26 16:12:47.023631 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.023605    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rlzvj_cab5fcb1-28bd-4602-937e-34b17c35871b/node-exporter/0.log"
Jul 26 16:12:47.046117 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.046044    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rlzvj_cab5fcb1-28bd-4602-937e-34b17c35871b/kube-rbac-proxy/0.log"
Jul 26 16:12:47.081859 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.081835    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rlzvj_cab5fcb1-28bd-4602-937e-34b17c35871b/init-textfile/0.log"
Jul 26 16:12:47.110819 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.110783    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-cjjgp_c4a05eea-40b1-4d16-b333-72d815bc2b6b/kube-rbac-proxy-main/0.log"
Jul 26 16:12:47.133262 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.133224    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-cjjgp_c4a05eea-40b1-4d16-b333-72d815bc2b6b/kube-rbac-proxy-self/0.log"
Jul 26 16:12:47.157244 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.157194    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-cjjgp_c4a05eea-40b1-4d16-b333-72d815bc2b6b/openshift-state-metrics/0.log"
Jul 26 16:12:47.193150 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.193121    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40f799a7-ce63-4379-b2c0-6302be099f1f/prometheus/0.log"
Jul 26 16:12:47.214830 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.214787    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40f799a7-ce63-4379-b2c0-6302be099f1f/config-reloader/0.log"
Jul 26 16:12:47.238559 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.238535    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40f799a7-ce63-4379-b2c0-6302be099f1f/thanos-sidecar/0.log"
Jul 26 16:12:47.260542 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.260513    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40f799a7-ce63-4379-b2c0-6302be099f1f/kube-rbac-proxy-web/0.log"
Jul 26 16:12:47.281341 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.281302    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40f799a7-ce63-4379-b2c0-6302be099f1f/kube-rbac-proxy/0.log"
Jul 26 16:12:47.302146 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.302069    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40f799a7-ce63-4379-b2c0-6302be099f1f/kube-rbac-proxy-thanos/0.log"
Jul 26 16:12:47.322351 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.322328    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40f799a7-ce63-4379-b2c0-6302be099f1f/init-config-reloader/0.log"
Jul 26 16:12:47.355222 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.355174    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-hr8zw_8d18e961-8a3e-469e-8357-cd70054a719d/prometheus-operator/0.log"
Jul 26 16:12:47.369120 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.369088    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-hr8zw_8d18e961-8a3e-469e-8357-cd70054a719d/kube-rbac-proxy/0.log"
Jul 26 16:12:47.547146 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.547115    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-659c57d745-72nj8_e738e949-e8ce-44a5-920a-00aacdb86c26/thanos-query/0.log"
Jul 26 16:12:47.601505 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.601470    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-659c57d745-72nj8_e738e949-e8ce-44a5-920a-00aacdb86c26/kube-rbac-proxy-web/0.log"
Jul 26 16:12:47.632662 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.632631    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-659c57d745-72nj8_e738e949-e8ce-44a5-920a-00aacdb86c26/kube-rbac-proxy/0.log"
Jul 26 16:12:47.660189 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.660154    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-659c57d745-72nj8_e738e949-e8ce-44a5-920a-00aacdb86c26/prom-label-proxy/0.log"
Jul 26 16:12:47.680554 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.680519    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-659c57d745-72nj8_e738e949-e8ce-44a5-920a-00aacdb86c26/kube-rbac-proxy-rules/0.log"
Jul 26 16:12:47.700966 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:47.700937    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-659c57d745-72nj8_e738e949-e8ce-44a5-920a-00aacdb86c26/kube-rbac-proxy-metrics/0.log"
Jul 26 16:12:48.693974 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:48.693923    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"]
Jul 26 16:12:48.698786 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:48.698755    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:48.706856 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:48.706795    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"]
Jul 26 16:12:48.832689 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:48.832652    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c72430bc-9396-43e7-a2c9-066327bbcad3-lib-modules\") pod \"perf-node-gather-daemonset-b9dc4\" (UID: \"c72430bc-9396-43e7-a2c9-066327bbcad3\") " pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:48.832960 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:48.832939    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c72430bc-9396-43e7-a2c9-066327bbcad3-sys\") pod \"perf-node-gather-daemonset-b9dc4\" (UID: \"c72430bc-9396-43e7-a2c9-066327bbcad3\") " pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:48.833113 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:48.833096    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c72430bc-9396-43e7-a2c9-066327bbcad3-proc\") pod \"perf-node-gather-daemonset-b9dc4\" (UID: \"c72430bc-9396-43e7-a2c9-066327bbcad3\") " pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:48.833249 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:48.833233    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c72430bc-9396-43e7-a2c9-066327bbcad3-podres\") pod \"perf-node-gather-daemonset-b9dc4\" (UID: \"c72430bc-9396-43e7-a2c9-066327bbcad3\") " pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:48.833378 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:48.833361    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwgbx\" (UniqueName: \"kubernetes.io/projected/c72430bc-9396-43e7-a2c9-066327bbcad3-kube-api-access-xwgbx\") pod \"perf-node-gather-daemonset-b9dc4\" (UID: \"c72430bc-9396-43e7-a2c9-066327bbcad3\") " pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:48.934528 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:48.934488    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c72430bc-9396-43e7-a2c9-066327bbcad3-proc\") pod \"perf-node-gather-daemonset-b9dc4\" (UID: \"c72430bc-9396-43e7-a2c9-066327bbcad3\") " pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:48.934722 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:48.934540    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c72430bc-9396-43e7-a2c9-066327bbcad3-podres\") pod \"perf-node-gather-daemonset-b9dc4\" (UID: \"c72430bc-9396-43e7-a2c9-066327bbcad3\") " pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:48.934722 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:48.934588    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwgbx\" (UniqueName: \"kubernetes.io/projected/c72430bc-9396-43e7-a2c9-066327bbcad3-kube-api-access-xwgbx\") pod \"perf-node-gather-daemonset-b9dc4\" (UID: \"c72430bc-9396-43e7-a2c9-066327bbcad3\") " pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:48.934722 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:48.934716    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c72430bc-9396-43e7-a2c9-066327bbcad3-lib-modules\") pod \"perf-node-gather-daemonset-b9dc4\" (UID: \"c72430bc-9396-43e7-a2c9-066327bbcad3\") " pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:48.934910 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:48.934757    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c72430bc-9396-43e7-a2c9-066327bbcad3-sys\") pod \"perf-node-gather-daemonset-b9dc4\" (UID: \"c72430bc-9396-43e7-a2c9-066327bbcad3\") " pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:48.934910 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:48.934869    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c72430bc-9396-43e7-a2c9-066327bbcad3-sys\") pod \"perf-node-gather-daemonset-b9dc4\" (UID: \"c72430bc-9396-43e7-a2c9-066327bbcad3\") " pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:48.935032 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:48.934929    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c72430bc-9396-43e7-a2c9-066327bbcad3-proc\") pod \"perf-node-gather-daemonset-b9dc4\" (UID: \"c72430bc-9396-43e7-a2c9-066327bbcad3\") " pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:48.935091 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:48.935055    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c72430bc-9396-43e7-a2c9-066327bbcad3-podres\") pod \"perf-node-gather-daemonset-b9dc4\" (UID: \"c72430bc-9396-43e7-a2c9-066327bbcad3\") " pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:48.935533 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:48.935498    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c72430bc-9396-43e7-a2c9-066327bbcad3-lib-modules\") pod \"perf-node-gather-daemonset-b9dc4\" (UID: \"c72430bc-9396-43e7-a2c9-066327bbcad3\") " pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:48.942825 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:48.942800    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwgbx\" (UniqueName: \"kubernetes.io/projected/c72430bc-9396-43e7-a2c9-066327bbcad3-kube-api-access-xwgbx\") pod \"perf-node-gather-daemonset-b9dc4\" (UID: \"c72430bc-9396-43e7-a2c9-066327bbcad3\") " pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:49.020729 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:49.020633    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:49.207334 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:49.207305    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"]
Jul 26 16:12:49.209804 ip-10-0-139-110 kubenswrapper[2559]: W0726 16:12:49.209754    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc72430bc_9396_43e7_a2c9_066327bbcad3.slice/crio-d76ae4e4da190115a82104c6249869d1bcc8ad2c712d9d3f85fde9987e3c400f WatchSource:0}: Error finding container d76ae4e4da190115a82104c6249869d1bcc8ad2c712d9d3f85fde9987e3c400f: Status 404 returned error can't find the container with id d76ae4e4da190115a82104c6249869d1bcc8ad2c712d9d3f85fde9987e3c400f
Jul 26 16:12:49.756298 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:49.756257    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4" event={"ID":"c72430bc-9396-43e7-a2c9-066327bbcad3","Type":"ContainerStarted","Data":"9a60a5e058695111e32099f96c3c5df5575c4cd0077752bdc4c160a9eb577bd8"}
Jul 26 16:12:49.756870 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:49.756354    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4" event={"ID":"c72430bc-9396-43e7-a2c9-066327bbcad3","Type":"ContainerStarted","Data":"d76ae4e4da190115a82104c6249869d1bcc8ad2c712d9d3f85fde9987e3c400f"}
Jul 26 16:12:49.756870 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:49.756801    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:49.774540 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:49.774488    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4" podStartSLOduration=1.774474068 podStartE2EDuration="1.774474068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 16:12:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:12:49.77325703 +0000 UTC m=+3254.063089150" watchObservedRunningTime="2026-07-26 16:12:49.774474068 +0000 UTC m=+3254.064306187"
Jul 26 16:12:50.022820 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:50.022740    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-5r4xv_43e1c401-0b5d-415c-ab0b-f91b4303dcf9/download-server/0.log"
Jul 26 16:12:50.669912 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:50.669882    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:12:50.728218 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:50.728170    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wjc9_d6d718b7-9f87-444a-af05-964167a2f945/konnectivity-agent/0.log" limit=1024
Jul 26 16:12:51.662617 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:51.662593    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-drfng_1a024411-b6f8-46e8-a695-84c440761063/dns/0.log"
Jul 26 16:12:51.683343 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:51.683317    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-drfng_1a024411-b6f8-46e8-a695-84c440761063/kube-rbac-proxy/0.log"
Jul 26 16:12:51.704930 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:51.704905    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-crzsx_72ff0d6f-05ae-4ce7-87b4-794a0d2c7bc9/dns-node-resolver/0.log"
Jul 26 16:12:52.337565 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:52.337533    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h8258_02874aa1-a6f9-47fa-af6f-373b4f475a65/node-ca/0.log"
Jul 26 16:12:53.415951 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:53.415924    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dc54b99bb-9tw42_1afad906-4350-4409-a472-c8b30df96f7c/kube-auth-proxy/0.log"
Jul 26 16:12:53.555664 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:53.555627    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-vc5dp_58a004c8-8db2-46f4-851a-fa381aa8024a/payload-pre-processing/0.log"
Jul 26 16:12:53.585233 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:53.585186    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wsp9v_afc1d831-651b-4136-8a8e-8440f9a966c3/payload-processing/0.log"
Jul 26 16:12:54.371331 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:54.371304    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5t4qj_487000fe-719e-4b3a-be94-d9d0cbcd94db/serve-healthcheck-canary/0.log"
Jul 26 16:12:55.065261 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:55.065232    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s7hbj_7d91f8c8-c702-4f74-a29e-af54043a79d5/kube-rbac-proxy/0.log"
Jul 26 16:12:55.084741 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:55.084715    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s7hbj_7d91f8c8-c702-4f74-a29e-af54043a79d5/exporter/0.log"
Jul 26 16:12:55.106998 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:55.106960    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s7hbj_7d91f8c8-c702-4f74-a29e-af54043a79d5/extractor/0.log"
Jul 26 16:12:55.775055 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:55.775023    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jncg9/perf-node-gather-daemonset-b9dc4"
Jul 26 16:12:57.981631 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:57.981595    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-84d47fc9db-j89xf_9b703ae5-cd78-4b6f-874c-240eee156343/manager/0.log"
Jul 26 16:12:58.326104 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:58.326069    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79d786d858-cz6m4_b9601f1f-c7fc-41a8-9336-c44781094551/manager/0.log"
Jul 26 16:12:58.342681 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:58.342650    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79d786d858-cz6m4_b9601f1f-c7fc-41a8-9336-c44781094551/copy-manifests/0.log"
Jul 26 16:12:59.334257 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:59.334229    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dc54b99bb-9tw42_1afad906-4350-4409-a472-c8b30df96f7c/kube-auth-proxy/0.log"
Jul 26 16:12:59.446083 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:59.446053    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-vc5dp_58a004c8-8db2-46f4-851a-fa381aa8024a/payload-pre-processing/0.log"
Jul 26 16:12:59.468747 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:59.468699    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wsp9v_afc1d831-651b-4136-8a8e-8440f9a966c3/payload-processing/0.log"
Jul 26 16:12:59.650991 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:12:59.650915    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7dd68d68f6-x7tvd_5ecd158c-4691-4c51-865a-09ecde033886/manager/0.log"
Jul 26 16:13:01.003927 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:13:01.003902    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dc54b99bb-9tw42_1afad906-4350-4409-a472-c8b30df96f7c/kube-auth-proxy/0.log"
Jul 26 16:13:01.118150 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:13:01.118118    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-vc5dp_58a004c8-8db2-46f4-851a-fa381aa8024a/payload-pre-processing/0.log"
Jul 26 16:13:01.142281 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:13:01.142248    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wsp9v_afc1d831-651b-4136-8a8e-8440f9a966c3/payload-processing/0.log"
Jul 26 16:13:05.800939 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:13:05.800859    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4d2ww_33a7f331-84f0-43c3-86fb-981f0fc567b9/kube-multus/0.log"
Jul 26 16:13:06.178087 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:13:06.178060    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wnzvq_c95dc61e-9fe1-40b4-85cf-edb31a0c5607/kube-multus-additional-cni-plugins/0.log"
Jul 26 16:13:06.197943 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:13:06.197913    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wnzvq_c95dc61e-9fe1-40b4-85cf-edb31a0c5607/egress-router-binary-copy/0.log"
Jul 26 16:13:06.217431 ip-10-0-139-110 kubenswrapper[2559]: I0726 16:13:06.217405    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wnzvq_c95dc61e-9fe1-40b4-85cf-edb31a0c5607/cni-plugins/0.log"